Valid data for school improvement final

John Cronin, Ph.D.
Vice President of Education Research
NWEA
john.Cronin@nwea.org
Using Valid Data to support School
Improvement Goals

NWEA
Is a not for profit assessment
organization that partners with over
7,000 school systems in the U.S. and
in 30 countries.
We are a leader in computer-
adaptive and formative assessment.

During this session you’ll
learn
• Why accurate valid tests are essential.
• That are three purposes for school
improvement goals.
• What it means to set reasonable goals.
• What to consider when writing a goal.

Three purposes of
school improvement
goals

Professional Development
Support teacher improvement by tying
professional development to learning in
the classroom

Professional Development
Key idea – The focus is on teacher
improvement, and the student learning
goal is simply one way to assess progress

Summative Evaluation
Use improvement goals as part of the
principal or teacher evaluation process

Key Idea – The goal should be at a level of
performance that would exceed that of
the likely “replacement educator”.

Aspirational Goals
A goal that puts students on track to
achieve their aspirations (college or
career)

Aspirational Goals
Key idea – Aspirational goals are not
always met, but they show students the
level of improvement needed for success.

Why accurate assessments
are essential

Computer adaptive tests are accurate for ALL students
Adult Literacy
Beginning Reading
Experience: low performing student
This student’s
instructional
level (50/50)
RIT Scale

Computer –adaptive tests are accurate for ALL students
Adult Literacy
Beginning Reading
Experience: High performing student
This
student’s
instructional
level (50/50)
RIT Scale

RTI Cut Score
(FSA Level 1)
RTI Identification – Student score is at NWEA 18th
percentile, cut score is at the NWEA 24th Percentile (FSA
Level 1 Cut Score)
Computer-adaptive test
standard error – 2.9
15% of students misidentified
The importance of accuracy in RTI placement

RTI Cut Score
(FSA Level 1)
RTI Identification – Student score is at NWEA 18th
percentile, cut score is at the NWEA 24th Percentile (FSA
Level 1 Cut Score)
Non-adaptive test with FSA
equivalent standard error at
Level 1 – 4.78
The importance of accuracy in RTI placement

Advanced math cut
score (FSA Level 5)
Advanced math identification – Student score is at NWEA 82nd
percentile, cut score is at the NWEA 87th Percentile (FSA Level 5
Cut Score)
Computer adaptive test
with standard error of 2.9
The importance of accuracy in Gifted Placement

Advanced math cut
score (FSA Level 5)
Advanced math identification – Student score is at NWEA 82nd
percentile, cut score is at the NWEA 87th Percentile (FSA Level 5
Cut Score)
Non-adaptive test with
standard error equivalent to
FSA Level 5 – 7.21
The importance of accuracy in Gifted Placement

190
195
200
205
210
215
220
Fall Term Spring Term
Standard error of math growth for student at FSA
Level 1 on NWEA MAP
Red Lines = Error of growth based on FSA standard
error at Level 1
Black Lines = Error of growth based on NWEA MAP
standard error at Level 1 equivalent.

220
225
230
235
240
245
250
Fall Term Spring Term
Standard error of math growth for student at FSA
Level 5 on NWEA MAP
Red Lines = Error of growth based on FSA standard
error at Level 5
Black Lines = Error of growth based on NWEA MAP
standard error at Level 5 equivalent.

If a teacher sets a high quality
goal, it is associated with
improved student achievement.
1
*Catalyst for change (2004), Community Training and Assistance Center, retrieved 10-2-13,
http://www.ctacusa.com/wp-content/uploads/2013/11/CatalystForChange.pdf
*It’s more than money (2013), Community Training and Assistance Center, retrieved 10-2-13,
http://www.ctacusa.com/wp-content/uploads/2013/11/MoreThanMoney.pdf

If a teacher sets and meets a goal,
it is associated with increased
student achievement.
2

Implementation of student
learning objectives can make a
12-13% difference in the growth
of student achievement.
3

When used for higher
stakes, teachers find
SLO’s threatening and
don’t trust goal setting
processes.

Teacher responses to the
Rhode Island SLO pilot
*Community Assistance and Training Center (2013, September) Focus on Rhode Island, Student Learning Objectives and Evaluation. http://www.ctacusa.com/wp-
content/uploads/2013/11/FocusOnRhodeIsland.pdf
Continuous performance
improvement is the core
intent of SLOs in teacher
evaluation
53%
20%
28%
Disagree Undecided Agree

SLO’s provide a credible
link between a teacher’s
craft and learning
outcomes. 77%
12%
10%

SLO’s improve student
achievement outcomes
71%
17%
12%

SLO’s provide sound
evidence for measuring
teacher performance
85%
9%6%

For most educators the focus is formative,
Goal setting should focus on
opportunities to learn and improve
and NOT on the outcome. When
goals are achieved, educators gain
confidence and become more
efficacious.

For most educators the focus is formative,
Tie student learning goals to
opportunities for teachers to learn
and grow. These may include peer
coaching and observation,
professional development
opportunities, and other strategies.

When the purpose is professional
development
• The primary goal is to connect teacher
professional development to the
learning of their students.
• The primary purpose is to help
teachers improve their practice, which
may lead to improved results
• When done well, this process
improves student learning.

Evaluate the performance of a school for
purposes of retaining the principal

For a small minority of educators the focus is
summative.
Goal setting challenges these
educators to improve their
performance and demonstrate their
competence.
Every educator should know which
category they are in!

In baseball, the goal of the general
manager is to hire players who all
hit .300. In reality, very few players
meet this aspirational goal.

Instead, general managers rely on a
statistic called WAR.

Players aren’t released because
they don’t hit .300, they are
released when a rookie or minor
leaguer would perform better.

Because new baseball managers
inherit their roster, their
accountability increases as their
control increases.

What happens when
educators are held
accountable to unreasonable
improvement goals?

They focus on
metrics rather
than learning

Serena is “one year” behind
proficiency for her grade
1.5
Her goal is to attain
years of growth this year.

45%
43%
39%
34%
31%
26%
21%
18%
16%
12%
9%
55%
57%
61%
66%
69%
74%
79%
82%
84%
88%
91%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
10
9
8
7
6
5
4
3
2
1
K
Proportion of students meeting 1.5 years of growth in
mathematics
Met 1.5 years Did not meet
Students start at 50th percentile of 2015 norms

Student growth across a school year
130
150
170
190
210
230
250
FALL WINTER SPRING NEXT FALL
AxisTitle
Mathematics
K 1 2 3 4 5 6 7 8 9

Student growth across a school year
FALL WINTER SPRING NEXT FALL
Reading
K 1 2 3 4 5 6 7 8 9

13%
9%
2%
1%
1%
1%
1%
1%
1%
1%
1%
87%
91%
98%
99%
99%
99%
99%
99%
99%
99%
99%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
10
9
8
7
6
5
4
3
2
1
K
Proportion of schools averaging 1.5 years of growth in
mathematics
Students start at 50th percentile of 2015 norms

44%
42%
38%
34%
30%
25%
21%
17%
14%
11%
6%
56%
58%
62%
66%
70%
75%
79%
83%
86%
89%
94%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
10
9
8
7
6
5
4
3
2
1
K
Proportion of low performing students meeting 1.5 years of
growth in mathematics
Students start at 20th percentile of 2015 school norms

9%
6%
1%
1%
1%
1%
1%
1%
1%
1%
1%
91%
94%
99%
99%
99%
99%
99%
99%
99%
99%
99%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
10
9
8
7
6
5
4
3
2
1
K
Proportion of schools with low performing students averaging
1.5 years of growth in mathematics
Students start at 20th percentile of 2015 school norms

What constitutes a reasonable
school improvement goal?

What makes a school improvement
goal realistic?
• What is the past track record of the
school?
• What is the range of improvement that
is normal for schools like this?
• What is the level of control that the
principal has over the conditions of
improvement?

It’s important to have a
“racing form” for student
achievement

This spring, 80% of my third grade students will meet
their growth projections according to their MAP
Reading scores.
Historical Growth: School Data
• 2014 45% of the students in the class
met their growth projections
• 2015 48% of the students in the class
met their growth projections

This spring, my students will show a median growth
percentile of 65%
Historical Growth: School Data
• 2014 The median growth percentile
was 57%
• 2015 The median growth percentile
was 52%

When the purpose is aspirational
• The primary purpose of the goal is to clarify
what level of performance is needed to
attain the aspired result.
• The goal isn’t necessarily realistic, because it
represents what is needed for success.
• The goal is never used summatively.

Ben wants to
attend the
University of
Florida. What
growth must he
show to qualify?

https://www.nwea.org/research/innovation-
leadership/research-data-galleries/mapping-the-road-to-
college/
Address of NWEA College Explorer
Tool

The goal – median ACT of the University of Georgia
freshman class
MAP Score = 260ACT Score = 29
Grade 7 Grade 8 Grade 9
244
250
256
235
244
253

What makes an improvement goal
realistic?
• What is the past track record of the school?
• What is the range of improvement that is
normal for schools like this?
• What is the level of control that the principal
has over the conditions of improvement?

The simple formula for setting
school improvement goals

All students should be in
play relative to the goal.
1

Fifth grade math achievement
in one school system
NUMBEROFSTUDENTS
FALL RIT
Mathematics
No Change Down Up

NUMBEROFSTUDENTS
STUDENT’S SCORE IN FALL
NUMBER OF 5TH GRADE STUDENTS MEETING PROJECTED GROWTH IN THE SAME DISTRICT
Mathematics
Below projected growth Met or above projected growth
Fifth grade math achievement
in one school system

The goal should always be
improvement in a subject.
2

Thank you!
E-mail: dawn.essig@nwea.org
Organization Website: www.nwea.org

Valid data for school improvement final

Recommended

Recommended

More Related Content

What's hot

What's hot (14)

Viewers also liked

Viewers also liked (12)

Similar to Valid data for school improvement final

Similar to Valid data for school improvement final (20)

More from John Cronin

More from John Cronin (20)

Recently uploaded

Recently uploaded (20)

Valid data for school improvement final

Editor's Notes