The Transition from MPG to MAP Deborah Adkins, Research Specialist, Leslie Yudman, ELA Content Specialist, NWEA Fusion 2012, the NWEA summer conference in Portland, Oregon This session addresses transitioning students from MPG to MAP. It examines the point at which most partners transition students and the nature of the effect on scores. Findings from a counterbalanced research study will be presented. The study was conducted with second graders who took MPG as first graders. These students were administered items from the 2-5 reading pool both with and without audio assistance. The session will conclude with proposed next steps. Learning Outcome: - Learn when to transition children from MPG to MAP

1. Transitioning
Students from
MPG to MAP

2. What will be covered?
• This session addresses transitioning
students from MPG to MAP.
• It examines the point at which most
partners transition students and the nature
of the effect on scores.
• Findings from a research study will be
presented.
• Next steps.
© 2011 Northwest Evaluation Association. All Rights Reserved. 2

3. When do most partners
transition students?
Most of our partners transition their
students from MPG to MAP
during the fall of 2nd grade for both
reading and mathematics.
© 2011 Northwest Evaluation Association. All Rights Reserved. 3

4. What is the impact ?
For reading an additional 9% of students experience
a drop in scores when transitioning from MPG to MAP
at the beginning of 2nd grade vs. the beginning of 3rd
grade
# lower %lower
Mean RIT
Total in fall in fall
diff
MPG to MAP SP10 to FA10 (RIT) (RIT)
(1 st-2 nd) 113,234 69,662 62% 10
MPG to MPG SP10 to FA10
(1 st-2 nd) 46,357 22,029 48% 6
MPG to MAP SP10 to FA10 (2 nd
-3 rd) 49,940 26,685 53% 9
MAP to MAP SP10 to FA10 (2 nd
-3 rd) 286,956 130,514 45% 7
© 2011 Northwest Evaluation Association. All Rights Reserved. 4

5. What is the impact ?
For mathematics the point of transition made little difference
Total # lower in %lower in Mean RIT
fall (RIT) fall (RIT) diff
MPG to MAP SP10 to FA10
(1 st-2 nd) 125,238 76,318 61% 8
MPG to MPG SP10 to FA10
(1 st-2 nd) 45,988 20,985 47% 5
MPG to MAP SP10 to FA10 (2 nd
-3 rd) 47,432 28,515 60% 7
MAP to MAP SP10 to FA10 (2 nd
-3 rd) 282,306 130,647 46% 6
© 2011 Northwest Evaluation Association. All Rights Reserved. 5

6. What does this mean?
There is something about the 2 forms of the
tests that is different.
• Item type
• Audio support for MPG
• Content
© 2011 Northwest Evaluation Association. All Rights Reserved. 6

7. Field Study (Fall 2011)
• The study was conducted with second graders
who took MPG as first graders.
• These students were administered items from
the 2-5 reading pool both with and without audio
assistance.
• Items in the field test were re-calibrated under
both the non-audio & audio conditions and
evaluated
• Field study tests were rescored
© 2011 Northwest Evaluation Association. All Rights Reserved. 7

8. Results of Fall 2011 Study
Students
the sample was similar to the population
MPG (spring grade 1)to MAP 2-5 (fall # lower %lower Mean RIT
Total
grade 2) in fall in fall diff
2010 - population 113,23 69,662 62% 10
4
2011 - study sample 1,161 814 70% 12
© 2011 Northwest Evaluation Association. All Rights Reserved. 8

9. Spring Grade 1 to Fall Grade 2
Transition
FA11
Q4 N = 3
FA11 FA11
Q3 N = 5 Q4 N = 8
FA11 FA11 FA11 FA11 FA11
Q1 N = 6 Q2 N = 30 Q3 N = 36 Q4 N = 50 >215 N = 6
FA11 Grade2 FA11 Grade2 FA11 Grade2 FA11 Grade2 FA11 Grade2 FA11 Grade2
<142 Q1 (142-165) Q2 (166-175) Q3 (176-186) Q4 (187-215) >215
N=2 N = 177 N = 108 N = 94 N = 173 N=0
SP11 Grade1 SP11 Grade1 SP11 Grade1 SP11 Grade1 SP11 Grade1 SP11 Grade1
<145 Q1 (145-166) Q2 (167-176) Q3 (177-186) Q4 (187-214) >214
N=8 N = 215 N = 318 N = 323 N = 290 N=7
FA11 FA11 FA11 FA11
Q1 N = 164 Q2 N = 85 Q3 N = 64 Q4 N = 7
FA11 FA11 FA11
<142 N = 2 Q1 N = 94 Q2 N = 30
FA11
Q1 N = 17
© 2011 Northwest Evaluation Association. All Rights Reserved. 9

10. Results of Fall 2011 Study
Items
Calibrations varied for 55-90% of the items
across all goals
Easier with Audio by More Difficult with Essentially the Same -
5+ RIT Audio by 5+ RIT within 4 RIT
Total
Number of Percentag RIT Percentag RIT Range
Goal Items e Range Dif e Dif Percentage
Informational
Comprehension 20 55.0% 5 to 17 0.0% NA 45.0%
Literary Comprehension 20 75.0% 6 to 24 15.0% 5 to 10 10.0%
Word Meaning &
Analysis 21 71.4% 6 to 22 4.8% 7 23.8%
+20 to 24 +15 to 19 +10 to 14 +5 to 9 +0 to 4 -1 to -4 -5 to -9 -10
Number of
Items 5 8 17 11 12 4 3 1
Baseline
RIT
Range 160-187 158-182 152-213 162-216 155-213 183-207 177-211 209
© 2011 Northwest Evaluation Association. All Rights Reserved. 10

11. Results of Fall 2011 Study
Items (cont.)
© 2011 Northwest Evaluation Association. All Rights Reserved. 11

12. Results of Fall 2011 Study
Calibrations by Goal - Informational Comprehension
© 2011 Northwest Evaluation Association. All Rights Reserved. 12

13. Results of Fall 2011 Study
Calibrations by Goal - Literary Comprehension
19
16
© 2011 Northwest Evaluation Association. All Rights Reserved. 13

14. Results of Fall 2011 Study
Calibrations by Goal - Word Meaning & Analysis
9
© 2011 Northwest Evaluation Association. All Rights Reserved. 14

15. Results of Fall 2011 Study
Field tests raw scores
The majority of students score higher on audio
across both forms of the test (72.1% form A &
78.5% form B) with only 9.4% & 7.2% yielding the
same score across forms respectively.
© 2011 Northwest Evaluation Association. All Rights Reserved. 15

16. Results of Fall 2011 Study
Field tests re-scored (scaled score)
Fall 2011 Grade 2 (MAP Reading 2-5)
Spring 2011 Grade 1(MPG Reading)
MAP 2-5 Audio FT NonAudio FT
Quartile RIT Range N RIT Range N N N
<1 <145 8 <142 4 0 0
1 145-166 215 142-165 458 30 179
2 167-176 318 166-175 253 124 301
3 177-186 323 176-186 199 383 366
4 187-214 290 187-215 241 621 315
>4 >214 7 >215 6 3 0
© 2011 Northwest Evaluation Association. All Rights Reserved. 16

17. Growth Targets
Growth Target Spring to Spring
Spring to Spring Growth Targets
Drop in Score from SP to
Overall No Drop in Score SP to FA
FA
Quartile
N Met Didn't Meet % Met N Met Didn't Meet % Met N Met Didn't Meet % Met
<1 8 5 3 62.5% 1 0 1 0.0% 7 5 2 71.4%
1 208 94 114 45.2% 138 46 92 33.3% 70 48 22 68.6%
2 313 159 154 50.8% 232 97 135 41.8% 81 62 19 76.5%
3 314 160 154 51.0% 231 95 136 41.1% 83 65 18 78.3%
4 281 127 154 45.2% 182 53 129 29.1% 99 74 25 74.7%
>4 7 2 5 28.6% 7 2 5 28.6% 0 0 0 0.0%
Total 1131 547 584 48.4% 791 293 498 37.0% 340 254 86 74.7%
© 2011 Northwest Evaluation Association. All Rights Reserved. 17

18. What does this mean?
• It is necessary to consider the purpose of the test
– MPG Reading: Assesses pre-and emergent readers
in grades K-2
– MAP 2-5 Reading: Assesses students in grades 2-5
who can read
• Tests administered to students for their intended
purpose provide information on the construct of
interest (e.g. performance on informational
comprehension, etc.). When tests are administered
otherwise, the likelihood of introducing construct
irrelevance increases.
© 2011 Northwest Evaluation Association. All Rights Reserved. 18

19. Transition Strategies
The strategy largely depends on
what you plan to do/decisions that
will be made based on the data in
conjunction with what works best
for your students
© 2011 Northwest Evaluation Association. All Rights Reserved. 19

20. Transition Strategies
Reading
• When the student can read connected text
sufficiently to access the items
• When the student has been exposed to grade
level content
• When the student scores at the upper end of
the scale for 2 consecutive administrations
© 2011 Northwest Evaluation Association. All Rights Reserved. 20

21. Transition Strategies
Mathematics
• When the student has been exposed to grade
level content
• When the student scores at the upper end of
the scale for 2 consecutive administrations
© 2011 Northwest Evaluation Association. All Rights Reserved. 21

22. Next Steps
• Concentrating on item development at the ends of the
scale
• Updated selection strategy for MPG Fall 2012
• Releasing Common Core aligned MPG for Fall 2012
• Additional studies
– Fall 2012 –MPG goal scores predict MAP scores
– Fall 2013 – Drop when transitioning from MPG Common
Core to MAP Common Core
– Fall 2013/Winter 2014 - MPG Common Core goal
scores predict MAP Common Core scores
• Research investigating use of a transition instrument
– To identify which test is more appropriate to students
© 2011 Northwest Evaluation Association. All Rights Reserved. 22

23. Thank You
Questions?
Contact information:
Transition Study : deborah.adkins@nwea.org
Product: kelly.goodrich@nwea.org
© 2011 Northwest Evaluation Association. All Rights Reserved. 23