www.earnperhit.com/essay => Professional academic writing www.lucky-bet.site => Bet on Sports - 50% Deposit Bonus www.lucky-bet.site/casino => Online Casino - 5000$ Welcome Bonus www.lucky-bet.site/lotto247 => Lotto247 - Win Big, Live Free www.lucky-bet.site/eurobet => Best European Bookmaker Presented at Washington Educational Research Association (WERA) conference. Presenters: Highline Public Schools and Vancouver Public Schools Sarah Johnson Sarah.Johnson@highlineschools.org Paul Stern Paul.Stern@vansd.org Presentation Overview: - Background/The Value of Alignment Studies - Highline’s Regression Study - NWEA’s Linking Study - Multi-District Regression Study - Conclusions - Applying the Results

1. Predicting Student Performance
on the MSP-HSPE
Understanding, Conducting, and Using
Alignment Studies
Highline Public Schools and
Vancouver Public Schools
Presenters:
Sarah Johnson Sarah.Johnson@highlineschools.org
Paul Stern Paul.Stern@vansd.org

2. Overview
• Background/The Value of Alignment Studies
• Highline’s Regression Study
• NWEA’s Linking Study
• Multi-District Regression Study
• Conclusions
• Applying the Results

3. About the MAP Assessments
• Computerized
• Adaptive assessment – increases in difficulty
when answers are correct and decreases in
difficulty when answers are incorrect.
• Rasch Units (RIT) Scale
• Equal Interval
• Vertical scaling
• Has the same meaning regardless of grade or
age of the student.
• For the purposes of this presentation, we will be
looking at Reading & Math only.

4. Value of Alignment Studies
Researchers align scales for one of two purposes:
• Use results from measure “X” to predict the value of
a harder-to-observe measure or outcome “Y”.
• Use results from measure “X” to predict the value of
a future measure or outcome “Y”.
In our case, faculty and administration are interested in
identifying students who are likely to struggle on future
state performance measures. By intervening early, we
can target resources to students who may not meet
“proficiency”.

5. http://kingsburycenter.org/gallery

6. About Vancouver Public Schools
• About 22,000 enrolled students
• 6 High Schools (4 comprehensive, 1 magnet, 1
alternative)
• 18% of students speak a language other than
English at home
• 53% eligible for free or reduced price lunch
• The district serves half of the city of
Vancouver, WA (across the river from Portland)

7. About Highline Public Schools
• About 18,000 enrolled students
• 15 High Schools (2 comprehensive, 6 small
learning community, 1 magnet, 5 alternative, 1
skills center)
• 43% of students speak a language other than
English at home - 21% ELL.
• 68% eligible for free or reduced price lunch
• The district serves neighborhoods of White
Center, Burien, Des Moines, SeaTac and
Normandy Park just south of Seattle.

8. Highline’s Regression Study
• In 2007, School and District Administration had
been requesting ways to interpret student MAP
scores in context of (then) WASL testing. One
concern in particular was that students had been
above average on the national norms, but yet
were not meeting standard on the state
assessment.
• School staff also requested a way to quickly
identify if a student was on track or not.

9. Highline’s Regression Study
• Decided to do a regression analysis to predict
WASL performance.
• Ran correlations on multiple variables, and
found that “HiMap” (max of last 3 test
administrations) had a higher correlation with
WASL than a single MAP score.
• Weeds out test “bombs” and missing data

10. “HIMAP” Variable Defined
Fall Winter SPRING
HIMAP HIMAP HIMAP
5th Grade 5th Grade 5th Grade

11. Highline’s Regression Study
• Rather than make a straight out prediction of
whether a student will meet/not meet
standard, we wanted to emphasize the possible
predictive error.
• Decided to find a cut on the MAP assessment to
predict 400 on WASL, and then generate an
error band around that where students would be
considered “too close to call”
• Used 4 points as a generous estimate of the
standard error of the assessment (usually
between 3-3.5)

12. Intervention Categories: 3 “Bands”
• “Above Benchmark” students were those who performed
more than 4 RIT points above the cut score. These
students are considered on track to meet standard.
• “Strategic” students were those who performed within 4
RIT points of the cut. These students are “too close to
call” and should receive strategic intervention to meet
standard.
• “Intensive” students were those who performed more
than 4 RIT points below the cut score. These students
are unlikely to meet standard without intensive
intervention.

14. Cuts for Fall, Winter and Spring
• When the study was first done in 2008, regression
analyses were performed using Spring MAP scores and
WASL.
• Growth norms were utilized to back track to get cuts for
Fall and Winter
• Cut scores and ranges were disseminated to teachers
and administrators, along with an explanation of the
scores.
• Excel files for schools began including MAP
scores, along with each students’ “BSI Indicator”, color
coded in Red, Yellow and Green.

17. Predictive Validity
• When a student’s indicator is compared to their actual
performance:
• Approximately 90% of students identified as “Above
Benchmark” actually met standard.
• Approximately 50% of students identified as
“Strategic” actually met standard.
• Approximately 10% of students identified as
“Intensive” actually met standard.
• These were generally true within about 10 percentage
points

19. 2010 MSP
• The analysis was re-run in 2010 following the
first year of transition from WASL to MSP.
• During the second analysis, regressions were
run on each test window individually in each
grade level, finding individual cuts, rather than
using growth norms.
• District budget cuts made high school MAP
testing optional, and therefore High School was
excluded.

21. 2012 MSP
• Due to online testing for MSP along with other
district initiatives requiring lab time, our Spring
window was moved from May to March
beginning in Spring 2012. Also, Winter testing
became an optional window.
• Therefore, cuts were created one more time in
2012 for Highline. Again, regressions were done
between Fall MAP and MSP, and Spring MAP
and MSP. Because Winter was optional, the cuts
for Winter were determined using the 2/3 point
between the Fall and Spring cuts.

23. NWEA’s Linking Study
• Most recently updated in Feb, 2011
• Based on a sample of 271 schools in the Spring
of 2010
• NWEA uses an Equi-percentile method to
equate test results

24. Equipercentile Method of
Alignment
• NWEA used a sample of students from 271 schools
taking the 2010 spring assessment in WA.
• For each grade and subject, identify the percentage
of students in the study sample that met standard.
• For each grade and subject, identify the RIT
associated with the equivalent percentile from within
the study sample.
“If 40% of the study population in grade 3 math
performed below the proficient level on the state
test, we would find the RIT score that would be
equivalent to the 40th percentile for the study
population”

29. Multi-District Regression Study
• Included 7 districts including
Seattle, Bellingham, Vancouver, Highline, Sumn
er, Auburn, and Clover Park
• Data covered the 2009-10 and 2010-11
academic years
• The “cut score” for proficiency was consistent
across both years at each grade level, so data
from both years was pooled
• Overall N of approximately 80,000

30. Independent Variables Created
• Math Spring RIT (Winter and Fall as well)
• Math Spring HIMAP (Winter and Fall as well)
• Combined Spring HIMAP (sum of Read & Math)
(Winter and Fall as well)
• Math Winter HIMAP + Math MSP
• Math Fall HIMAP + Math MSP
(Comparable variables were also created for Reading)

31. Quality of Correlation
Best: (Corr: 0.78)
• Spring RIT (but no predictive value, so Spring
indicators will be ignored)
Next Best: (Corr: 0.73-0.75)
• Winter RIT
• Winter HIMAP + MSP scale score (275-500)
• Winter HIMAP
Third Best: (Corr: 0.70)
• Read Winter HIMAP + Math Winter HIMAP

32. Rationale for Selecting Winter HIMAP
• Spring MAP test window overlaps MSP/HSPE
test window.
• Prior Year MSP scores not available for grades 3
and 10.
• New students in district are missing MSP
scores.
• Not all students perform to their best ability on
every test.
• Many students do not take the Winter MAP.

33. Rationale for Selecting Winter HIMAP
Winter HIMAP …
• Is not very different in the quality of the
correlation as compared to other options,
• Maximizes the number of students for whom it
can be applied, and
• Is relatively easy to explain

36. Predictive Validity: Percent of
Students Meeting Standard by Band
Multi-
10%-20% of
READING District NWEA Highline “Likely Not
Likely Not Proficient 15% 17% 20% Proficient”
Students Met
At Risk 55% 59% 65% Standard.
Likely Proficient 92% 93% 94%
50%-65% of “At
Risk Students Met
Multi- Standard.
MATH District NWEA Highline
90%-95% of
Likely Not Proficient 10% 14% 14% “Likely Proficient”
At Risk 50% 62% 61% Students Met
Likely Proficient 92% 95% 95% Standard.

37. Pro and Con of Do-It-Yourself
Pro:
• Data are based on “our kids” (this is an emotional
argument, not a statistical one).
• Winter and prior spring estimates can be computed
rather than estimated.
Con:
• It is a lot of work.
• Controlling for test windows is complex.
• NWEA results are very similar to DIY results.
• Teachers who encounter the NWEA linking study will be
confused about why our cut points are different.
• … and did I say it was a lot of work?

41. Read Met MSP MSP Math Met MSP MSP
Growth Read 2013 2013 Read Growth Math 2013 2013 Math
LAST FIRST Target Read Fall Read Read Spring Target Math Fall Math Math Spring
ID# NAME NAME Grade 2012 Fall RIT Pctile Categ. Odds Target 2012 Fall RIT Pctile Categ. Odds Target
10944 5 Yes 202 34 46% 207 No 196 12 17% 203
13455 5 No 202 34 46% 207 Yes 208 40 36% 216
13980 5 Yes 215 73 75% 219 No 228 88 81% 235
17713 5 No 217 78 80% 221 No 215 61 52% 223
17716 5 No 192 14 24% 199 Yes 184 3 5% 192
17719 5 No 206 45 54% 211 No 204 29 27% 212
17728 5 Yes 211 61 66% 215 Yes 208 40 36% 216
17732 5 Yes 213 67 70% 217 Yes 208 40 36% 216
17736 5 Yes 216 76 78% 220 Yes 227 87 80% 234
17804 5 Yes 203 36 48% 208 Yes 217 66 56% 224
18312 5 Yes 205 42 52% 210 Yes 201 21 22% 208
18328 5 Yes 212 64 68% 216 Yes 202 24 24% 210
18578 5 No 203 36 48% 208 Yes 201 21 22% 208
18624 5 Yes 216 76 78% 220 Yes 206 34 32% 214
19057 5 No 225 93 89% 228 Yes 212 52 46% 220
19128 5 Yes 217 78 80% 221 Yes 222 78 68% 229
21036 5 No 176 2 5% 186
24125 5 No 215 73 75% 219 Yes 210 46 40% 218
26414 5 No 194 17 27% 201 Yes 209 43 38% 217
27807 5 No 180 4 8% 189 Yes 185 3 5% 193
30737 5 No 205 42 52% 210 No 209 43 38% 217
36075 5 No 201 31 43% 206 No 185 3 5% 193
36376 5 Yes 171 1 3% 182 191 7 9% 199
41166 5 No 184 6 12% 192 No 197 14 18% 204
43584 5 No 230 97 93% 233 Yes 224 82 72% 231
46978 5 Yes 197 22 34% 203 Yes 197 14 18% 204