We share a new and novel analysis of state and regional trends with a focus on bright spots – where we are seeing progress that can help all schools and systems improve faster. We hope this analysis is a resource for all of us working to increase access to educational opportunities for our most vulnerable children, and that it helps us individually and collectively allocate our time and resources to make the greatest impact possible.

1. Pandemic Learning Recovery
in St. Louis
February 2024
Analysis and visualization by

2. PAGE 2
Welcome & Agenda
1. State and Regional Education Data Analysis
2. Insights from Fast-Improving Systems
3. Public Data Tool

3. PAGE 3
Zoom Etiquette and Engagement
• Please mute when not speaking to avoid background noise
interruptions.
• We love to see everyone’s beautiful faces! However, please turn
off your camera while tending to any interruptions to avoid
distracting others.
• We’ll pause after the analysis for discussion. Please share any
questions and comments in the chat throughout.

4. PAGE 4
Meet Steve Cartwright
Steve Cartwright
Founder, Exponent Education
• Early research experience at Urban Institute
and DC Public Schools
• Served as Fellow, Strategic Data Project,
Harvard Center for Education Policy
Research
• Analytics consultant with schools, systems,
and support orgs across the country

5. PAGE 5
Today’s content
1 Pandemic impact on student achievement
2 New evidence on regional “rebounding”
3 Finding and learning from outliers

6. PAGE 6
Recent national data from Harvard and Stanford - as featured
in the NYT - show the extent of Covid’s impact on achievement

7. PAGE 7
The results illustrate two different stories of Covid’s impact
and students’ learning “recovery”
• Prior to the pandemic,
students in the richest districts
in the country were 2 grade
levels above the national
average and the pandemic
affected these students by only
about one half of one grade
level
• In the poorest schools, where
students were already more
than 1 year behind, the
pandemic reduced
performance by nearly one
grade level

8. PAGE 8
In St. Louis, students lost nearly 3 years of learning and - as of
2022 - were cumulatively 5 years behind the national average

9. PAGE 9
Other high poverty districts serving a majority of Black students
were impacted less by the pandemic and are recovering

10. PAGE 10
Today’s content
1 Pandemic impact on student achievement
2 New evidence on regional “rebounding”
3 Finding and learning from outliers

11. PAGE 11
ELA performance flatlined 2022 to 2023 statewide and
improved modestly in St. Louis city and county
Interact with these data on www.theopportunitytrust.org
• Statewide, ELA achievement did not move significantly from ‘22 to ’23.
+1 +1
• Locally, both charters and SLPS improved modestly (+1% point each).
• No sector has reached pre-pandemic levels of performance.
+0 +1

12. PAGE 12
• Nationally and locally, students lost more ground in mathematics than reading.
Math proficiency is recovering more quickly than ELA, but
remains low and behind pre-pandemic levels
Interact with these data on www.theopportunitytrust.org
+1 +1 +2 +3
• Continued ”rebounding” exists in math: students statewide, in St. Louis county, and
across the city are improving, through no sector is back to pre-pandemic achievement.

13. PAGE 13
• Selective admissions schools perform 5-7x better than open enrollment schools, which
enroll the vast majority of SLPS students.
Aggregate results mask a large inequality of outcomes
between open enrollment and selective admissions schools
5x
larger
7x
larger
• Selective admissions schools have seen greater rates of improvement year-to-year and
have fully recovered from the pandemic (meaning 2023 performance exceeds 2019).

14. PAGE 14
Since 2019, the Black-white achievement gap expanded
significantly in St. Louis
• Statewide and in the county, gaps narrowed modestly in ELA and expanded modestly in
math.
• These gaps widened significantly in St. Louis city, by 8-12% points depending on the sector
and subject.
• Mirroring findings on selective schools, the Black-white gap in SLPS is more than 50%
points in both subjects.
-2
-2
+2
+11
+8
+11
+12
+0

15. PAGE 15
Twenty years of research identifies key milestones in
children's development that predict meaningful life outcomes
Kindergarten
readiness
3rd grade
reading
proficiency
8th grade
math
proficiency
4th grade
math
proficiency
A local study found that
students entering
kindergarten behind their
peers continued to lag 18
months later (University of
Missouri 2019). Another
study from Ohio found nearly
three in four (74%) students
who tested positively for a
potential problem on a
kindergarten readiness
assessment scored below
the passing promotion score
on the third grade ELA
assessment (Justice et al
2019).
Students who were not
proficient in third grade
reading are four times as
likely not to graduate from
high school as those who
were reading proficiently in
third grade. For children
who lived in a high-poverty
neighborhood for at least a
year and were not reading
proficiently in third grade,
they are six times as likely
not to graduate (Hernandez
2012).
Researchers studying ten-
year-old students’
mathematical knowledge in
the US and the UK found that
their mastery of fractions
was highly predictive of their
overall mathematical and
algebraic knowledge in high
school, five to six years after
initial testing (Siegler et al
2012).
Researchers consider eighth
grade math the
“gatekeeper” to higher math
and science courses and
other postsecondary
opportunities for both
college and career track
students (Wang and
Goldsmith 2003).

16. PAGE 16
Interact with these data on www.theopportunitytrust.org
• Since the pandemic, the percent of charter students scoring below basic - the lowest of
four categories - has increased 11% points.
• Both sectors have more than 50% of 3rd grade students below basic, the lowest level.
Milestone: 3rd grade reading proficiency
• The 3rd grade results mirror the overall trend of some modest improvements locally.
• In SLPS, nearly 60% of students are below basic, but this category reduced by -6% points
from ‘22 to ‘23.

17. PAGE 17
Interact with these data on www.theopportunitytrust.org
Milestone: 8th grade math proficiency
• In SLPS, that figure expanded by 7% points and is now more than 7 in 10 students.
• Unlike 3rd grade, the charter sector has reduced the percentage of students below basic
by -2% points since the beginning of the pandemic.
• Statewide and in St. Louis county, the percentage of below basic students actually
increased from 2022 to 2023.

18. PAGE 18
Today’s content
1 Pandemic impact on student achievement
2 New evidence on regional “rebounding”
3 Finding and learning from outliers

19. PAGE 19
The schools best helping students recover aren’t just those
increasing proficiency rates
Interact with these data on www.theopportunitytrust.org
• Proficiency rates are
crude measures to
use when evaluating
change over time.
Math Performance by Level
Black Students
• Movement across all
four performance
levels provides a more
detailed view of
change.
• Unlike the state, EW
Kaufmann increased
proficient and
advanced rates and
significantly
decreased the rate of
students below basic
over two years.

20. PAGE 20
Fast-recovering local schools/systems for Black students
include a mix of districts and charters in the St. Louis area
• Outlier schools achieve outsized performance index changes with Black students and
done so consistently from 2021 to 2022 and from 2022 to 2023.
Interact with these data on www.theopportunitytrust.org
Note: Only schools with <10% of students omitted are shown.
Compute a performance index to
capture movement across all
levels of performance
Identify systems with large,
consistent gains from 2021 to
2022 and 2022 to 2023

21. Compared to the state, county, and open enrollment SLPS
schools, outliers are improving 2x-15x faster
PAGE 21
Interact with these data on www.theopportunitytrust.org
Note: Only schools with <10% of students omitted are shown.
+2.7 index points +7.0 index points +28.7 index points
+15.8 index points +32.6 index points
+1.9 index points

22. 15
11
24
34
0
10
20
30
40
50
60
70
80
90
100
ELA Math
2023 Actual 2028 Projection
PAGE 22
Outliers present an example of what’s possible citywide when
the right conditions for improvement are in place
St. Louis City Black Student Proficiency Rate Projection
SLPS and Charter
• Current proficiency rates for
Black students citywide are
15% in ELA and 11% in math.
* +1.8% points in ELA and +4.7% points in math.
**See Chetty, R., Friedman, J. N., & Rockoff, J. (2011). New evidence on the long-term impacts of tax credits. IRS white paper.
In an average class of 30 students, this amounts to 1
additional student proficient in ELA and 2 additional
students proficient in math per year.
Statewide all student average
44
41
• If every SLPS school and
charter improved their
proficiency rates for Black
students at University City’s
rate* for the next five years,
citywide Black student
achievement would
dramatically improve.
• These increased test scores
translate into significant
improvements in college
attendance and early career
earnings.**

23. PAGE 23
Necessary conditions for improvement
Essential
Conditions
Key Components
Missouri’s
Status
Strong
Governance
• Sets ambitious but feasible goals for improvement
• Provides support and resources to Executive and team to create a
strong plan
• Holds Executive accountable to progress against goals
Effective
Leadership
• Recruits and retains high performing team
• Builds a goal-oriented, performance culture
• Develops team members
• Takes personal responsibility for outcomes
Robust
Instructional Core
• Research-based curriculum materials aligned to grade-level
expectations
• Teacher planning protocols
• Principal and teacher observation and feedback
• Data-driven improvement cycles
Accountability
and Choice
• Transparent data and reporting
• Accountability that rewards and recognizes progress and
intervenes where there is persistent failure
• Choice with dollars following students

24. PAGE 24
Continue to track the region’s recovery using our
online data tools
learning.
theopportunitytrust.org

25. PAGE 25
“We are confronted with the fierce urgency of now…
Now let us begin. Now let us rededicate ourselves to the long and
bitter -- but beautiful -- struggle for a new world.”
Thank you!