1. RESEARCH POSTER PRESENTATION DESIGN © 2011
www.PosterPresentations.com
The troubling issue of the SAT Superscore as an institutional metric
Mark A. Davenport
University of North Carolina at Greensboro
Joe’s superscore is 1080 based on high
section scores with tests 1 and 2. His
superscore can go up but never go
down, regardless of further retakes. In
effect, all error is assumed to
underestimate his superscore, never
overestimate it.
We graphed Joe’s four SAT scores below
in the order he took the tests.
According to classical test theory, as his
SAT Composite average score goes up or
down with each retake, his average
(about 988) will approach a better
estimate of his true aptitude.
Some institutions cite changes in cohort
SAT score averages as evidence that
efforts to improve applicant quality
have worked (or fallen short). This
assumes that SAT aggregate scores are
mostly analogous to individual scores:
That is, changes in average SAT scores
reflect changes in the ‘average
aptitude’ of the cohort. However, hybrid
scoring schemes such as the
‘Superscore’ may introduce some
troubling confounds for aggregate score
users.
Superscores are an issue only for students who take the SAT more than once. Students
that took the test multiple times had slightly higher high school GPAs than did those who
took it once (< 5-hundredths of a grade point from 2006 to 2009). However, their first SAT
Composite was far lower, hinting that students believed the SAT underestimated their
aptitude. The charts below demonstrate the inflationary effect of superscoring on the
otherwise lower scores of the retake students.
Superscoring clearly benefits students who are trying to enter college.
We found no evidence that
superscoring improves prediction.
After accounting for high school
GPA, superscores were no better
than other scoring methods at
predicting 1st year college GPA.
In the aggregate, superscoring is not so benign. The profound upward bias shown in
superscores occurs only for those who take the SAT more than once. As the percent of
students retaking the SAT fluctuates, the cohort SAT average will fluctuate accordingly.
At UNCG, the percent of students with retakes has varied from 54% to 69% of the yearly
freshman cohort since 2001. With superscoring, the average SAT for any cohort takes on an
added ‘behavioral’ dimension not intentioned by the creators of the test.
Users of the SAT Superscore should:
• Understand that standard
psychometric assumptions do not
apply.
• Accept that the aggregate superscore
has a ‘non-cognitive’ component that
makes it inherently unstable.
• Consider using individual superscores
for admissions but other scores, such
as the SAT Composite average, for
aggregate reporting.
Crocker, L. & Algina, J. (1986).
Introduction to Classical and Modern
Test Theory. Orlando, FL: Holt,
Rinehart & Winston.
Davenport, M. (2010). Special report to
the Director of Admissions on the
instability of SAT scores.
Greensboro, NC: UNCG.
Mark A. Davenport, Ph.D.
Senior Research Associate
Office of Institutional Research
356 McIver Bldg
University of North Carolina At Greensboro
Greensboro, NC 27402-6170
M_Davenport@uncg.edu