More Related Content
Similar to Oerc va sessions dec 12 2013 final for post
Similar to Oerc va sessions dec 12 2013 final for post (20)
More from Ohio Education Research Center
More from Ohio Education Research Center (20)
Oerc va sessions dec 12 2013 final for post
- 5. Session Objectives
Provide value-added information relevant to
the practitioner’s perspective
What
should teachers and principals know and
understand about SAS® EVAAS® value-added
reporting?
How may educators use SAS EVAAS valueadded information to inform practice and
impact student academic progress?
© 2013, Battelle for Kids. All Rights Reserved.
- 6. Value-Added Measures in Ohio
For more than 10 years, Battelle for Kids has
provided support for professional learning and
fostered collegial dialogue for understanding
and using value-added measures.
Battelle for Kids continues to advocate for the
use of value-added measures to inform
practices and support student learning.
© 2013, Battelle for Kids. All Rights Reserved.
- 7. Ohio’s Value-Added Models
Generated by SAS® Analytics:
Customers in 135 countries
More than 65,000 business, government
and university sites
SAS customers or their affiliates
represent 90 of the top 100 companies
on the 2012 FORTUNE Global 500® list
© 2013, Battelle for Kids. All Rights Reserved.
- 8. Value-Added Information in Practice:
Building Awareness
Understanding the Difference
© 2013, Battelle for Kids. All Rights Reserved.
Achievement Measures
Progress/Growth Measures
- 10. Measuring Growth is Important for
ALL Students
Proficient
3rd
4th
5th
6th
Grade
© 2013, Battelle for Kids. All Rights Reserved.
7th
8th
- 11. Why can’t we simply compare OAA scaled scores
from one year to the next to measure growth?
Spring 2013 OAA Math Scaled Score Ranges
Level
Advanced
3rd
4th
5th
6th
7th
8th
447-521 452-551 439-548
448-575
458-574 459-537
Accelerated 429-446 432-451 424-438
429-447
436-457 432-458
Proficient
400-428 400-431 400-423
400-428
400-435 400-431
Basic
378-399 377-399 382-399
378-399
378-399 379-399
Limited
240-377 239-376 245-381
235-377
268-377 288-378
© 2013, Battelle for Kids. All Rights Reserved.
- 12. Value-Added Information in Practice:
SAS EVAAS MRM Model
Mutlivariate Response Model (MRM)
© 2013, Battelle for Kids. All Rights Reserved.
Grades 4–8 Reading and Math
Ohio Achievement Assessments
- 13. SAS® EVAAS ® Value-Added:
Two Models in Ohio
Mean Gain Approach (MRM)
4–8
Reading and Math (OAA) Reports
Predicted Mean Approach (URM)
5
& 8 Science (OAA) School and District Reports
Extended Testing
ODE RttT Mini-Grant
BFK SOAR
© 2013, Battelle for Kids. All Rights Reserved.
- 14. SAS® EVAAS ® MRM Model*
Uses grades 3–8 Reading and Math OAAs
Compares
the average growth of students in
the most recent year to the average growth of
students in 2010 (state’s baseline year).*
Growth
expectation is defined as maintaining
placement in the distribution of NCE scores
from one year to the next.*
*conceptual level
© 2013, Battelle for Kids. All Rights Reserved.
- 15. Raw Score
Scaled Score
Sample
Raw Score
Range
Sample
Scaled Score
Range
52
551
NCE
Normal Curve
Equivalent
NCE
99
1
0
© 2013, Battelle for Kids. All Rights Reserved.
247
- 16. Scaled Scores Converted to NCEs in
2010 Baseline Year
Conversion Values Are Fixed/Frozen
Example
2010 Scaled Scores
Rank Ordered
551
Normal Curve Equivalent
(NCE)
247
1
© 2013, Battelle for Kids. All Rights Reserved.
99
- 17. Value-Added Terminology
Normal Curve Equivalent (NCE)
The NCE is similar to a percentile rank in that
scores are derived from scaled scores and
ranked based upon performance.
A significant difference between percentile rank
and NCE is that an NCE scale is an equal
interval scale.
© 2013, Battelle for Kids. All Rights Reserved.
- 18. Normal Curve Equivalent (NCE)
Distribution
of Scores
Percentile
Equivalents
Normal Curve
Equivalents
© 2013, Battelle for Kids. All Rights Reserved.
1
1
5
10
10
20
20 30 40 50 60 70 80 90
30
40
50
60
70
95
80
99
90
99
- 19. Value-Added Terminology
Baseline Score
Group of students’ prior year mean NCE
Example: Spring 2012 OAA mean NCE
Observed Score
of students’ new/most recent mean NCE
Example: Spring 2013 OAA mean NCE
Group
© 2013, Battelle for Kids. All Rights Reserved.
- 20. Basic, Conceptual Example
Scaled scores are converted to NCEs
Grade 6
Baseline
Grade 7
Observed
394 = 46
Student 1
430 = 59
402 = 50
Student 2
417 = 54
384 = 42
Student 3
400 = 49
394 = 46
Student 4
390 = 44
410 = 52
Student 5
425 = 57
Mean Baseline = 47.2
Mean Observed = 52.6
Growth = Mean Observed – Mean Baseline
Growth = 52.6 – 47.2 = 5.4 (Mean NCE Gain)
A basic measure of the growth for this group is 5.4 NCEs
© 2013, Battelle for Kids. All Rights Reserved.
- 21. Teacher Value-Added Report
Note: Battelle for Kids is utilizing visual representations of
copyrighted EVAAS® Web reporting software from SAS in this
presentation for instructional purposes.
© 2013, Battelle for Kids. All Rights Reserved.
- 22. Statistical Terminology
Standard Error
All
measures of student learning contain error.
Standard error is a measure of the
uncertainty.
In the EVAAS teacher value-added report, the
size of the standard error is influenced by
N size (size of the student group)
Missing scores
© 2013, Battelle for Kids. All Rights Reserved.
- 23. Value-Added Terminology
Growth Index
Since
the size of the standard error (degree of
certainty) will vary across teachers, their
estimated gain must be standardized to include
both the estimate and the degree of certainty
(standard error).
Divides
a teacher’s estimated gain by the
associated standard error.
© 2013, Battelle for Kids. All Rights Reserved.
- 25. Levels of Teacher Value-Added Effects
Most
Effective
Teacher's index: 2 or greater
Students are making substantially more progress than the state
growth standard.
Above
Average
Teacher's index: equal to or greater than 1, but less than 2
Students are making more progress than the state growth
standard.
Average
Teacher's index: equal to or greater than -1, but less than 1
Students are making the same amount of progress as the state
growth standard.
Approaching Teacher's index: equal to or greater than -2, but less than -1
Students are making less progress than the state growth standard.
Average
Least
Effective
© 2013, Battelle for Kids. All Rights Reserved.
Teacher's index: less than -2
Students are making substantially less progress than the state
growth standard.
- 27. Using Value-Added to Inform Practice
Key Considerations:
Systemic
Programs, Delivery Models, Structures,
Services, etc.
Access
to Professional Learning
Curriculum
Instruction
Assessment
© 2013, Battelle for Kids. All Rights Reserved.
- 28. Link/Roster Verification Process
Ohio’s Journey:
Spring
2011
Spring
2012
Spring
2013
30%
60%
100%
of Ohio’s
LEAs
Participated
© 2013, Battelle for Kids. All Rights Reserved.
of Ohio’s
LEAs
Participated
of Ohio’s
LEAs
Participated
- 29. What is Roster Verification?
A process to capture the instructional linkage
between teachers and students.
Teachers verify three questions for each class
roster:
1. I taught these students
2. During these months
3. For __% of instruction (sole or shared)
© 2013, Battelle for Kids. All Rights Reserved.
- 30. Why Do We Link?
Student schedule files do not always capture
an accurate connection between teachers and
the students they teach.
Roster Verification provides teachers and
principals with a solution to review, modify,
and verify class rosters to ensure the
instructional linkage between teachers and
students is accurate.
© 2013, Battelle for Kids. All Rights Reserved.
