The document provides an analysis of trends in student academic achievement in Missouri following the Covid-19 pandemic. Some key points: - Statewide and in Missouri, average test scores declined significantly from 2019 to 2022, especially in math. Low-income students in Missouri experienced some of the largest declines nationally. - While proficiency rates remained flat or continued declining in most areas, some schools ("outliers") saw significant improvements, especially in reducing the percentage of students scoring below basic. Fast-improving schools included both districts and charters serving mostly low-income students. - If all schools could improve as quickly as the highest performing outliers, statewide proficiency among low-income students could increase dramatically by 2027.

1. Learning to Improve:
A First Look at Trends and Bright Spots
in School Systems Performance
Post-Pandemic
State of Missouri
January 2023
Analysis and visualization by

2. PAGE 2
Welcome & Agenda
1. State and Regional Education Data Analysis
2. Insights from Fast-Improving Systems
3. A New Public Data Tool and Improvement Resource

3. PAGE 3
Meet Steve Cartwright
Steve Cartwright
Founder, Exponent Education
• Early research experience at Urban Institute and
DC Public Schools
• Served as Fellow, Strategic Data Project, Harvard
Center for Education Policy Research
• Analytics consultant with schools, systems, and
support orgs across the country

4. PAGE 4
Today’s content
1 Pandemic impact on national and state achievement
2 New evidence on regional achievement
3 Finding and learning from outliers

5. PAGE 5
• Losses in these key
milestones ranged from
about half to nearly all
of the gains made
between 2000 and
2019.
The Covid-19 pandemic significantly impacted student
academic achievement across the country
272
276 278
280 282 283 284
281 282 281
273
211
216 217
220 220 220 221 221 221 219
216
2000 2003 2005 2007 2009 2011 2013 2015 2017 2019 2022
National Assessment of Educational Progress
Composite Score, 2009-2022, All Students
-8 points
(loss of 86% of
2000 to 2019 gains)
-3 points
(loss of 41% of
2000 to 2019 gains)
Grade 8 Math
Grade 4 Reading
No Child Left Behind:
national emphasis on
assessment and
accountability
Race to the Top: national
emphasis on standards
and teacher evaluation
Current era
interrupted by
global pandemic
• Average achievement
increased significantly
between 2000 and 2013
before stagnating and
declining.
• The pandemic most
affected students’ math
performance, but may
have lingering effects in
reading.
Information on NAEP cut scores for Basic and Proficient performance can be found here.
Grade 8 Proficient
Grade 8 Basic
Grade 4 Basic
Grade 4 Proficient

6. PAGE 6
• Low-income students in
Missouri were
disproportionately
affected and
experienced some of
the nation’s largest
declines.
In Missouri, average declines were similar to the nation, but
losses for low-income students were particularly large
222 221 221
224
220
222 223 223
218
213
208 209 208
210
207
211 211 211
207
197
279
276
281
286
282 283
281 281 281
272
263 262
266
272
269
271
266 268 267
256
2003 2005 2007 2009 2011 2013 2015 2017 2019 2022
National Assessment of Educational Progress - Missouri
Composite Score, 2009-2022
-9 points (loss of >400% of
2003 to 2019 gains)
-11 points (loss of >250% of
2003 to 2019 gains)
-5 points (loss of ~110%
greater than 2003 to 2019
loss)
-10 points (loss of >1100%
of 2003 to 2019 loss)
Grade 8 Math
Grade 4 Reading
• Low-income students in
Missouri lost more
ground than their peers
in most other states.
All Students
Low-Income Students
All Students
Low-Income Students

7. PAGE 7
Grade 4 Reading Composite Score
NAEP State Rankings for Low-Income Students
Grade 8 Math Composite Score
NAEP State Rankings for Low-Income Students
Missouri’s low-income students declined more than their peers
in most states and now perform near the bottom nationally

8. PAGE 8
Today’s content
1 Pandemic impact on national and state achievement
2 New evidence on regional achievement
3 Finding and learning from outliers

9. PAGE 9
• Urban districts:
KCPS and SLPS
• Urban charter:
Charter LEAs operating in Kansas City and St. Louis City
• Suburban counties:
“Charter” counties of Clay, St. Charles, St. Louis, Jefferson, and Jackson (minus
KCPS)
• Semi-urban counties:
The 13 “first class” counties like Cole, Cape Girardeau, etc.
• Rural counties:
All others
Districts’ results are aggregated into one of five mutually
exclusive groupings

10. PAGE 10
ELA performance continued to decline from 2021 to 2022
statewide and remained relatively flat locally
Interact with these data on www.theopportunitytrust.org
• Statewide and in most geographies, ELA proficiency continued to decline from ‘21 to ‘22.
+1 +2
• Both urban districts and charters improved modestly (+1% and +2% points, respectively).
• No region or sector has reached pre-pandemic levels of performance.

11. PAGE 11
• Mirroring national trends, students lost more ground in mathematics than reading.
Math proficiency shows some signs of “rebounding”
statewide, unlike ELA
Interact with these data on www.theopportunitytrust.org
+4 +5 +8 +6
• Some signs of ”rebounding” exist in math: students in every geography and sector are improving
to varying degrees, through no sector is back to pre-pandemic achievement.
+3 +3

12. PAGE 12
Breaking down the urban results highlights some differences
across sectors
+1 +2 +3 -1 +6 +3 +9 +7
• In ELA, gains were modest in both St. Louis and Kansas City, with KCPS declining slightly.
• In math, the charter sector rebounded at 2x the rate of district schools in St. Louis and 1.3x the
rate of district schools in Kansas City

13. PAGE 13
Twenty years of research identifies key milestones in
children's development that predict meaningful life outcomes
Kindergarten
readiness
3rd grade
reading
proficiency
8th grade
math
proficiency
4th grade
math
proficiency
A local study found that
students entering kindergarten
behind their peers continued to
lag 18 months later (University
of Missouri 2019). Another
study from Ohio found nearly
three in four (74%) students
who tested positively for a
potential problem on a
kindergarten readiness
assessment scored below the
passing promotion score on the
third grade ELA assessment
(Justice et al 2019).
Students who were not
proficient in third grade reading
are four times as likely not to
graduate from high school as
those who were reading
proficiently in third grade. For
children who lived in a high-
poverty neighborhood for at
least a year and were not
reading proficiently in third
grade, they are six times as
likely not to graduate
(Hernandez 2012).
Researchers studying ten-year-
old students’ mathematical
knowledge in the US and the
UK found that their mastery of
fractions was highly predictive
of their overall mathematical
and algebraic knowledge in
high school, five to six years
after initial testing (Siegler et al
2012).
Researchers consider eighth
grade math the “gatekeeper”
to higher math and science
courses and other
postsecondary opportunities
for both college and career
track students (Wang and
Goldsmith 2003).

14. PAGE 14
Interact with these data on www.theopportunitytrust.org
• Nearly two thirds (63%) of urban district students scored below basic - the lowest of four levels
of performance.
• In Missouri’s cities, more than 50% of 3rd grade students score at the lowest level of performance.
Milestone: 3rd grade reading proficiency
• The 3rd grade results mirror the overall trend of stagnation statewide with some modest
improvements locally.

15. PAGE 15
Interact with these data on www.theopportunitytrust.org
Milestone: 8th grade math proficiency
• Urban districts and charters significantly reduced the percentage of below basic students.
• “Rebounding” in math is shown not just in proficiency improvements, but also in reductions in
the percentage of students below basic statewide.

16. PAGE 16
Today’s content
1 Pandemic impact on national and state achievement
2 New evidence on regional achievement
3 Finding and learning from outliers

17. PAGE 17
The schools best helping students recover aren’t just those
increasing proficiency rates
Interact with these data on www.theopportunitytrust.org
• Proficiency rates are
crude measures to use
when evaluating change
over time.
48%
40%
26%
42%
19%
14%
7%
4%
2022
2021
Basic Below Basic Proficient Advanced
43%
42%
28%
27%
21%
21%
8%
10%
2022
2021
Basic Below Basic Proficient Advanced
ELA Performance by Level, 2021 to 2022
Low-Income Students
Missouri Brookside Charter (KC)
No change
-2% pts.
+1% pt.
+1% pt.
+5% pts.
+3% pts.
+8% pts.
-16% pts.
• Movement across all
four performance levels
provides a more
detailed view of change.
• Unlike the state,
Brookside Charter
increased proficiency
rates and decreased
the rate of students
below basic.

18. PAGE 18
Fast-recovering schools/systems for low-income students
include a mix of districts and charters
• Outlier schools achieve outsized performance index changes with low-income students (and serve
a majority of low-income students overall).
Interact with these data on www.theopportunitytrust.org
Note: Only schools with <10% of students omitted are shown.
School composition at
least 50% Low Income
Compute a performance
index to capture
movement across all
levels of performance

19. Outliers are improving significantly faster than district and
county averages
PAGE 19
Interact with these data on www.theopportunitytrust.org
Note: Only schools with <10% of students omitted are shown.
Rural Counties
Brookside Charter Zalma R-V
+29.6 index points
+27.2 index points
Missouri
-4.4 index points -2.0 index points -5.2 index points
Kansas City 33

20. PAGE 20
Outliers present an example of what’s possible when the right
conditions for improvement are in place
29
25
80
64
0
10
20
30
40
50
60
70
80
90
100
ELA Math
2022 Actual 2027 Projection
Missouri Low-Income Student Proficiency Rate Projection
• Current proficiency rates for
low-income students are 29% in
ELA and 25% in math.
* +10.2% points in ELA and +7.9% points in math.
**See Chetty, R., Friedman, J. N., & Rockoff, J. (2011). New evidence on the long-term impacts of tax credits. IRS white paper.
In an average class of 30 students, this amounts to 3
additional students proficient in ELA and 2 additional
students proficient in math per year.
• If every school improved their
proficiency rates for low-
income students at Zalma’s
rate* for the next five years,
statewide low-income student
achievement would
dramatically improve.
• These increased test scores
translate into significant
improvements in college
attendance and early career
earnings.**
Statewide all student average
43
39

21. PAGE 21
Necessary Conditions for Improvement
Essential
Conditions
Key Components
Strong
Governance
• Sets ambitious but feasible goals for improvement
• Provides support and resources to Executive and team to create a strong plan
• Holds Executive accountable to progress against goals
Effective
Leadership
• Recruits and retains high performing team
• Builds a goal-oriented, performance culture
• Develops team members
• Takes personal responsibility for outcomes
Robust
Instructional
Core
• Research-based curriculum materials aligned to grade-level expectations
• Teacher planning protocols
• Principal and teacher observation and feedback
• Data-driven improvement cycles
Accountability
and Choice
• Transparent data and reporting
• Accountability that rewards and recognizes progress and intervenes where there is
persistent failure
• Choice with dollars following students