The document discusses aligning local benchmark assessments with high-stakes state tests through a process of descriptive and inferential statistical analysis. It provides an example analysis of alignment between a district's 3rd-5th grade ELA fall benchmark scores and state test scores. A strong, statistically significant correlation was found between the two sets of scores. Based on the analysis, the district revised cut scores for performance levels to better align local benchmarks with state test expectations. The process led to increased confidence in conclusions about student performance and improved organizational integrity.

1. Aligning Benchmarks with High Stakes Assessments Devin Vodicka, Ed.D.

2. Outline Focus is on alignment after results have been obtained. This can be called a “back-end” alignment approach. Context Data-Driven Decision-Making Understanding Data Descriptive Analysis Inferential Analysis Impact Reflection

3. The Context: Carlsbad Unified School District Currently in Year 3 Data Director Implementation 14 Schools & 11,000 ADA 9 K-5 3 Middle Schools 1 High School 1 Alternative School District API: 2007: 829 2008: 843 2009: 858 Eight Title I Schools 0 schools in PI

4. What is Data-Driven Decision-Making? http://www.portical.org/ http://www.clrn.org/elar/

5. Understanding Data Variables Numeric: Continuous scale Attribute: Discontinuous scale Descriptive Analysis Use visual displays such as charts and graphs Numeric Central Tendency, Standard Deviation Attribute: Frequency Tables Inferential Analysis Determines if relationships between variables are “statistically significant” Examples: Chi-Square, ANOVA, Regression Analysis

6. Descriptive Analysis Create Test Series Use Pre-Built Reports Pivot Table Create Custom Reports

7. 3 rd Grade ELA Fall Benchmark Alignment 2009 2008

8. 6 th Grade ELA 1 st Semester Benchmark Alignment 2008 2009

9. Custom Reports

10. Inferential Analysis Download Excel data from Custom Reports Import into Stats Program NCSS Three Calculations: Attribute to Attribute (Chi Square) Attribute to Numeric (ANOVA) Numeric to Numeric (Regression)

11. Fall ELA Alignment Attribute Alignment Chi-Square Statistics Section Probability Level 0.000000 Reject H0

12. Fall ELA Alignment Analysis of Variance Table Prob Level 0.000000* * Term significant at alpha = 0.05

13. Fall ELA Alignment

14. Fall Summary Statement The equation of the straight line relating X2008_CST_ELA_Scaled_Scores and X_4th___5th_Grade_ELA_Fall__Total_Pe is estimated as: X2008_CST_ELA_Scaled_Scores = (185.8394) + (2.9938) X_4th___5th_Grade_ELA_Fall__Total_Pe using the 743 observations in this dataset. The y-intercept, the estimated value of X2008_CST_ELA_Scaled_Scores when X_4th___5th_Grade_ELA_Fall__Total_Pe is zero, is 185.8394 with a standard error of 5.4943. The slope, the estimated change in X2008_CST_ELA_Scaled_Scores per unit change in X_4th___5th_Grade_ELA_Fall__Total_Pe, is 2.9938 with a standard error of 0.0829. The value of R-Squared, the proportion of the variation in X2008_CST_ELA_Scaled_Scores that can be accounted for by variation in X_4th___5th_Grade_ELA_Fall__Total_Pe, is 0.6376. The correlation between X2008_CST_ELA_Scaled_Scores and X_4th___5th_Grade_ELA_Fall__Total_Pe is 0.7985. A significance test that the slope is zero resulted in a t-value of 36.1041. The significance level of this t-test is 0.0000. Since 0.0000 < 0.0500, the hypothesis that the slope is zero is rejected. The estimated slope is 2.9938. The lower limit of the 95% confidence interval for the slope is 2.8313 and the upper limit is 3.1563. The estimated intercept is 185.8394. The lower limit of the 95% confidence interval for the intercept is 175.0709 and the upper limit is 196.6080.

15. What does this mean? There is a statistically-significant relationship between 5 th Grade Fall ELA Performance Levels and CST ELA Performance Levels. The correlation between the 5 th Grade Fall ELA Percentage Correct and the CST ELA Scale Scores is about 80%. 64% of the variation in CST Scale Scores can be predicted by the Fall ELA Percentage Correct. CST Scale Score = 185 + (2.99 x %)

16. 2 nd -5 th Grade Summary

17. Calculating Cut-Points Create algebra formula for % (Scale – Intercept) / ( Slope) = % Feed Scale Scores into formula

18. 5 th Grade Revised Cut Scores 75% + Adv 59% - 74% Prof 43% - 58% Basic 33% - 42% BB 0 – 32% FBB Spring 80% + Adv 63% - 79% Prof 45% - 62% Basic 33% - 44% BB 0 – 32% FBB Mid 71% + Adv 56% - 70% Prof 39% - 55% Basic 29% - 38% BB 0 - 28% FBB Fall 5 Percentage Bands Performance Level Term Grade

19. 4 th Grade Revised Cut Scores 76% + Adv 62% - 75% Prof 45% - 61% Basic 34% - 44% BB 0 – 33% FBB Spring 77% + Adv 62% - 76% Prof 44% - 61% Basic 34% - 43% BB 0 – 33% FBB Mid 71% + Adv 53% - 70% Prof 37% - 52% Basic 25% - 36% BB 0 – 24% FBB Fall 4 Percentage Bands Performance Level Term Grade

20. 3 rd Grade Revised Cut Scores 85% + Adv 68% - 84% Prof 51% - 67% Basic 37% - 50% BB 0 – 36% FBB Spring 89% + Adv 71% - 88% Prof 55% - 70% Basic 42% - 54% BB 0 – 41% FBB Mid 79% + Adv 60% - 78% Prof 41% - 59% Basic 26% - 40% BB 0 – 25% FBB Fall 3 Percentage Bands Performance Level Term Grade

21. 2 nd Grade Running Records Fall Spring

22. 2 nd Grade Revised Cut Scores 9-11 Adv 6-8 Prof 4-5 Basic 2-3 BB 0-1 FBB Spring 86% + Adv 71% - 85% Prof 57% - 70% Basic 46% - 56% BB 0 – 45% FBB Mid 6 + Adv 4-5 Prof 3 Basic 1-2 BB 0 FBB Fall 2 Percentage Bands Performance Level Term Grade

23. Middle School ELA: Writing Prompts 41% Yes 2 nd 25% Yes 1 st 8 12% Yes to CST Writing Cluster 29% Yes to CST Overall 2 nd 10% Yes to CST Writing Cluster 22% Yes to CST Overall 1 st 7 30% Yes 2 nd 23% Yes 1 st 6 R-Squared Significant Relationship? Semester Grade

24. Impact Presentations & Feedback Teacher Leaders Principals Revised Cut Scores Performance Level Descriptors http://www.cde.ca.gov/ta/tg/sr/documents/pldreport.pdf Increased Confidence in Conclusions Improvement in Organizational Integrity?

25. Next Steps Math Alignment K-8 New Adoptions implemented in 2009-10 High School Alignment English Math Social Studies Science Identify and Promote “Best Practices” Grades

26. Reflection In your environment, how aligned are your local assessments with the high-stakes tests? How do you know? How could you find out? What would be the impact in your district of going through an alignment analysis?

27. Conclusion Questions? [email_address] Check out www.acsa.org and then search for “Vodicka” to find article “ Building Trust Through Data” (with Lisa Gonzales)