SlideShare a Scribd company logo
1 of 46
Download to read offline
City Study
2022:
MEMPHIS
Table of Contents
0 1 R E PORT OVE R VIE W
• About The City Studies Project
• Sectors of Schools
• Research Question and Analyses
• Measure of Academic Performance
Student Subgroup Analysis
Summary of Findings
0 3 APPE N DIX E S
• Acknowledgments
• Types of Charter Schools
• Methods
• Days of Learning
0 2 R E SE AR CH
F IN DIN G S
• Reading & math
Overall Memphis Results
Sector Analysis
• vs. state & comparison
within Memphis
Charter Subsector Analysis
Research Findings Cont’d.
Black Students
• vs. state & comparison
within Memphis
Hispanic Students
• vs. state & comparison
within Memphis
Students in Poverty
• vs. state & comparison
within Memphis
ELL Students
• vs. state & comparison
within Memphis
Special Ed Students
• vs. state & comparison
within Memphis
Male Students
• vs. state & comparison
within Memphis
Female Students
• vs. state & comparison
within Memphis
• vs. state
• comparison within Memphis
REPORT OVERVIEW
01
About The City Studies Project
Cohort 1 Cohort 2
The City Studies project examines the performance of schools in select U.S. cities,
including Memphis. We study the academic progress of students as the measure of school
performance.
O
O
C
C
CHARTER SCHOOLS
Public schools operated independently from the
traditional school district, with autonomy in adapting
school designs and held accountable for education
results.
Charter Management Organizations (CMOs)
Organizations holding the charter and overseeing the
operation of at least three charter schools.
Independent Charter Schools
Organizations holding the charter and overseeing the
operation of a single or two charter schools.
ACHIEVEMENT SCHOOL DISTRICT (ASD) SCHOOLS
Schools needing improvement that are directly run by
the district’s staff or operated by charter management
organizations (CMOs) authorized by the district.
OTHER DISTRICT-RUN SCHOOLS
Public schools not belonging to any of above types.
C
Sectors of Schools
COMMUNITIES MAY HAVE UP TO FOUR SECTORS OF SCHOOLS
OPTIONAL SCHOOLS
District-run schools that screen all the students with
academic criteria.
Research Question and Analyses
IN THIS REPORT WE EXAMINE ACADEMIC PERFORMANCE IN
MEMPHIS USING DATA FROM THE SCHOOL YEARS 2016-17
THROUGH 2018-19. THERE ARE THREE LEVELS OF ANALYSIS.
Overall performance in
Memphis schools over two
years.
• The performance of Memphis
students is benchmarked against the
state average performance,
accounting for student
characteristics.
• The performance of ASD school
students, performance of charter
school students, performance of
optional school students within
Memphis are then compared to that
of similar traditional public school
(other district school) students
within Memphis.
Performance for ASD
schools, charter
schools, optional
schools, and the rest of
Memphis public schools
over two years
Performance in the 2018-
2019 school year by
school type, race,
poverty status, English
language learner (ELL)
status, special
education status and
gender.
WE MAKE TWO SETS
OF COMPARISONS.
01 02 03
Achievement scores capture what a student knows at a point
in time. They are influenced by students’ prior conditions in
addition to schools’ contributions.
Growth scores indicate how much progress a student makes
from one year to the next. Growth scores allow us to zero in
on the contributions of schools separately from other factors
that affect point-in-time scores.
ACHIEVEMENT VS. GROWTH
We analyze student growth in standard deviation units so
that the results can be assessed for statistical differences.
In the following tables of findings, we transform growth
from standard deviation units into days of learning based on
a typical 180-day school year.
IN THIS STUDY WE MEASURE
ACADEMIC PERFORMANCE AS HOW
MUCH GROWTH STUDENTS MAKE FROM
ONE YEAR TO THE NEXT
Measure of Academic Performance
RESEARCH FINDINGS
02
-250
-200
-150
-100
-50
0
50
100
150
200
250
SY 2017-2018 SY 2018-2019
Growth
(in
Days
of
Learning)
significantly different at p< 0.05
reading math
Research Findings > Overall Memphis Results
> Reading & Math
Average One-Year Learning Gains for All Memphis
Students Compared to the State Average Learning
Gains, by Year and Subject
-250
-200
-150
-100
-50
0
50
100
150
200
250
SY 2017-2018 SY 2018-2019
Growth
(in
Days
of
Learning)
Learning Gains in Reading for Students in Memphis
Charter Schools, Memphis ASD Schools, Memphis
Optional Schools and Memphis District Compared to
the State Average Learning Gains by Year
significantly different at p< 0.05
charter innovation
Research Findings > Sector Analysis
> Reading
VS. STATE & COMPARISON WITHIN MEMPHIS
charter innovation district
optional
Reading
Charter vs. District
Optional vs. District
ASD vs. District
Charter vs. Optional
Charter vs. ASD
'17-'18 '18-'19
Tests of Differences
-250
-200
-150
-100
-50
0
50
100
150
200
250
SY 2017-2018 SY 2018-2019
Growth
(in
Days
of
Learning)
significantly different at p< 0.05
charter innovation district
Research Findings > Sector Analysis
> Math
VS. STATE & COMPARISON WITHIN MEMPHIS
Learning Gains in Math for Students in Memphis
Charter Schools, Memphis ASD Schools, Memphis
Optional Schools and Memphis District Compared to
the State Average Learning Gains by Year
optional
Math
Charter vs. District
Optional vs. District
ASD vs. District
Charter vs. Optional
Charter vs. ASD
Tests of Differences
'17-'18 '18-'19
-250
-200
-150
-100
-50
0
50
100
150
200
250
CMO Charter Schools Independent Charter Schools
Growth
(in
Days
of
Learning)
Relative Learning Gains for Students in Memphis
Network-Affiliated Charter Schools and Independent
Memphis Charter Schools Compared to the Average
Learning Gains for All Student in the State, by
Subject
significantly different at p< 0.05
reading math
Research Findings > Charter Subsector Analysis
> vs. state & comparison within Memphis
Reading
CMOs vs Independent Charter Schools
Math
CMOs vs Independent Charter Schools
sig
Tests of Differences
-600 -500 -400 -300 -200 -100 0 100 200 300 400 500
Growth (in Days of Learning)
Charter
District
Optional
ASD
Research Findings > School-Level Performance by Sector
> Reading
-600 -500 -400 -300 -200 -100 0 100 200 300 400 500
Growth (in Days of Learning)
Charter
District
Optional
ASD
Research Findings > School-Level Performance by Sector
>Math
-150
-100
-50
0
50
100
150
200
250
Memphis Black Students
Growth
(in
Days
of
Learning)
Learning Gains for All Memphis Black Students
Compared to the Average Black Learning Gains of
Black Students Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Black Students
ALL VS. STATE
-150
-100
-50
0
50
100
150
200
250
Memphis Charter
Black Students
Memphis
Optional
Black Students
Memphis District
Black Students
Memphis ASD
Black Students
Growth
(in
Days
of
Learning)
Learning Gains for Black Charter School Students,
Black Innovation School Students, and Black District
Students in Memphis Compared to the Average
Learning Gains of Black Students Statewide, by
Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Black Students
VS. STATE BY SECTOR
Reading
Charter Black vs. District Black
Optional Black vs. District Black
ASD Black vs. District Black
Math
Charter Black vs. District Black
Optional Black vs. District Black
ASD Black vs. District Black
Tests of Differences
sig
-150
-100
-50
0
50
100
150
200
250
Memphis Hispanic Students
Growth
(in
Days
of
Learning)
Learning Gains for All Memphis Hispanic Students
Compared to the Hispanic Average Learning Gains of
Hispanic Students Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Hispanic Students
ALL VS. STATE
-150
-100
-50
0
50
100
150
200
250
Memphis Charter
Hispanic
Students
Memphis
Optional
Hispanic
Students
Memphis District
Hispanic
Students
Memphis ASD
Hispanic
Students
Growth
(in
Days
of
Learning)
Learning Gains for Hispanic Charter School Students,
Hispanic Innovation School Students, and Hispanic
District Students in Memphis Compared to the
Average Learning Gains of Hispanic Students
Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Hispanic Students
VS. STATE BY SECTOR
Reading
Charter Hispanic vs. District Hispanic
Optional Hispanic vs. District Hispanic
ASD Hispanic vs. District Hispanic
Math
Charter Hispanic vs. District Hispanic
Optional Hispanic vs. District Hispanic
ASD Hispanic vs. District Hispanic
Tests of Differences
sig
-150
-100
-50
0
50
100
150
200
250
Memphis Students in Poverty
Growth
(in
Days
of
Learning)
Learning Gains for All Memphis Students in Poverty
Compared to the Poverty Average Learning Gains of
Students in Poverty Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Students in Poverty
ALL VS. STATE
-150
-100
-50
0
50
100
150
200
250
Memphis Charter
Students in
Poverty
Memphis
Optional
Students in
Poverty
Memphis District
Students in
Poverty
Memphis ASD
Students in
Poverty
Growth
(in
Days
of
Learning)
Learning Gains for Memphis Charter School Students
in Poverty, Memphis Innovation School Students in
Poverty, and Memphis District Students in Poverty
Compared to the Average Learning Gains of Students
in Poverty Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Students in Poverty
VS. STATE BY SECTOR
Reading
Charter Poverty vs. District Poverty
Optional Poverty vs. District Poverty
ASD Poverty vs. District Poverty
Math
Charter Poverty vs. District Poverty
Optional Poverty vs. District Poverty
ASD Poverty vs. District Poverty
Tests of Differences
sig
-150
-100
-50
0
50
100
150
200
250
Memphis ELL Students
Growth
(in
Days
of
Learning)
Learning Gains for All ELL Students in Memphis
Compared to the Average ELL Learning Gains of ELL
Students Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> ELL Students
ALL VS. STATE
-150
-100
-50
0
50
100
150
200
250
Memphis Charter
ELL Students
Memphis
Optional
ELL Students
Memphis District
ELL Students
Memphis ASD
ELL Students
Growth
(in
Days
of
Learning)
Learning Gains for ELL Students in Memphis Charter
Schools, ELL Students in Memphis Innovation
Schools, and ELL Students in Memphis District
Compared to the Average Learning Gains of ELL
Students Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> ELL Students
VS. STATE BY SECTOR
Reading
Charter ELL vs. District ELL
Optional ELL vs. District ELL
ASD ELL vs. District ELL
Math
Charter ELL vs. District ELL
Optional ELL vs. District ELL
ASD ELL vs. District ELL
Tests of Differences
sig
Learning Gains for All Memphis Students in Special
Education Compared to the Average Learning Gains
of Students in Special Education Statewide, by
Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Special Ed Students
ALL VS. STATE
-150
-100
-50
0
50
100
150
200
250
Memphis Special Ed Students
Growth
(in
Days
of
Learning)
-150
-100
-50
0
50
100
150
200
250
Memphis Charter
Students in
Special Ed.
Memphis
Optional
Students in
Special Ed.
Memphis District
Students in
Special Ed.
Memphis ASD
Students in
Special Ed.
Growth
(in
Days
of
Learning)
Learning Gains for Memphis Charter School Students
in Special Ed., Memphis Innovation School Students
in Special Ed., and Memphis District Students in
Special Ed. Compared to the Average Learning Gains
of Students in Special Ed. Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Special Ed Students
VS. STATE BY SECTOR
Reading
Charter Sped vs. District Sped
Optional Sped vs. District Sped
ASD Sped vs. District Sped
Math
Charter Sped vs. District Sped
Optional Sped vs. District Sped
ASD Sped vs. District Sped
Tests of Differences
sig
-150
-100
-50
0
50
100
150
200
250
Memphis Male Students
Growth
(in
Days
of
Learning)
Learning Gains for All Memphis Male Students
Compared to the Average Male Learning Gains of
Male Students Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Male Students
ALL VS. STATE
-150
-100
-50
0
50
100
150
200
250
Memphis Charter
Male Students
Memphis
Optional
Male Students
Memphis District
Male Students
Memphis ASD
Male Students
Growth
(in
Days
of
Learning)
Learning Gains for Male Charter School Students,
Male Innovation School Students, and Male District
Students in Memphis compared to the Average
Learning Gains of Male Students Statewide, by
Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Male Students
VS. STATE BY SECTOR
Reading
Charter Male vs. District Male
Magnet Male vs. District Male
Innovation Male vs. District Male
Math
Charter Male vs. District Male
Magnet Male vs. District Male
Innovation Male vs. District Male
Tests of Differences
sig
Reading
Charter Male vs. District Male
Optional Male vs. District Male
ASD Male vs. District Male
Math
Charter Male vs. District Male
Optional Male vs. District Male
ASD Male vs. District Male
Tests of Differences
sig
-150
-100
-50
0
50
100
150
200
250
Memphis Female Students
Growth
(in
Days
of
Learning)
Learning Gains for All Memphis Female Students
Compared to the Female Average Learning Gains of
Female Students Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Female Students
ALL VS. STATE
-150
-100
-50
0
50
100
150
200
250
Memphis Charter
Female Students
Memphis
Optional
Female Students
Memphis District
Female Students
Memphis ASD
Female Students
Growth
(in
Days
of
Learning)
Learning Gains for Female Charter School Students,
Female Innovation School Students, and Female
District Students in Memphis Compared to the
Average Learning Gains of Female Students
Statewide, by Subject
significantly different at p< 0.05
reading math
Research Findings > Student Subgroup Analysis
> Female Students
VS. STATE BY SECTOR
Reading
Charter Female vs. District Female
Optional Female vs. District Female
ASD Female vs. District Female
Math
Charter Female vs. District Female
Optional Female vs. District Female
ASD Female vs. District Female
Tests of Differences
sig
Summary of Findings
The summary of the findings from the analysis of Memphis schools is presented here.
APPENDIXES
03
Acknowledgments
Tennessee Department of
Education and Tennessee
Education Research Alliance
provided supports in the acquisition
of student-level data.
Shelby County Schools
assisted CREDO with
verifying the list of public
schools in Memphis.
Types of Charter Schools
• With more schools and students than a single charter
school, CMOs have some operational advantages in their
ability to spread administrative fixed costs, thus
providing the possibility of greater efficiency. In
addition, CMOs may be able to support additional
programs and more robust staffing.
• Whether CMOs lead to better student outcomes is a
matter of interest across the country.
OUR ANALYSES OF MEMPHIS CHARTER
SCHOOLS INCLUDE A BREAKOUT OF
CMOS AND INDEPENDENT CHARTERS.
There are two types
of charter schools.
CHARTER MANAGEMENT ORGANIZATIONS (CMOS)
Organizations holding the charter and overseeing the
operation of at least three charter schools.
INDEPENDENT CHARTER SCHOOLS
Organization holding the charter and overseeing the
operation of a single charter school. It may run the school
directly or contract with an organization which provides
services to one or two charter schools.
Methods
The annual academic growth of students in Memphis from
2016-17 to 2018-19, overall and by sector, is benchmarked
to the state average growth, accounting for student
characteristics.
We also explore how one-year growth of Memphis students
for the period ending in Spring 2019 differs by school type,
race, poverty status, English language learner status,
special education status, and gender.
Days of Learning
While these tools create precise and reliable answers,
they are presented in technical terms that are not
user-friendly to a general audience. To translate the
technical results into terms that are accessible to non-
technical audiences, CREDO developed Days of
Learning.
CREDO USES ADVANCED
TECHNOLOGY AND
SOPHISTICATED STATISTICAL
TOOLS TO MEASURE STUDENTS,
SCHOOLS AND THE EDUCATION
LANDSCAPE.
Think about the students in your state’s public schools. For many of their years
of schooling, they take achievement tests to measure what they know at the end of
the school year. We can identify the average score for each test each year.
Imagine a student who scores exactly at the average in one year, say 4th
grade, and then in the following year, scores exactly at the average again on the 5th-
grade test. The amount of year-to-year learning for that student show us what the
average learning is for all the students who took both tests.
We do that calculation for every grade the state tests: 4th to 5th, 5th to 6th,
and so on.
CREDO uses those annual measures of average learning to represent a typical
year of learning, and equates that to a typical 180-day school year. We say that the
student in our example has gained 180 days of learning.
If a student makes more progress than the average student, we take the
amount of extra achievement and translate it into 180-days of learning plus “X” extra
days. We are creating a measure of student learning as if the student went to school
for 180 days plus X days. The size of “X” depends on how much more the student
learns than the average student — if it’s a lot more, then “X” will be a large number,
and if it’s a small amount more, “X” will be a small number.
The same is true for students who do not learn as much as the average
student. Instead of adding to the 180-days-of-learning average, we subtract from
that base to reflect the smaller-than-average advances that those students realize. In
these cases, the difference leads to numbers such a “165 days of learning” or “152
days of learning”. Against the average standard of 180 days, these smaller days show
that students learned as if they had only attended school for 180 days minus X days
during the school year.
01
02
03
04
05
06
3rd
Grade
4th
Grade
= 180
days
More than 180
days
Less than 180
days
Student
A
Student
A
Overall Memphis Results
Significant at p < 0.05*
Significant at p < 0.01**
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
M e m p h i s O v e r a l l 2 0 1 7 - 1 8 0.00 -3 -0.02 -10
M e m p h i s O v e r a l l 2 0 1 8 - 1 9 -0.03* -19* -0.01 -7
Memphis School Sectors Compared
to State Average
Significant at p < 0.05*
Significant at p < 0.01**
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
A S D S c h o o l s 2 0 1 7 - 1 8 -0.09** -52** -0.08* -50*
A S D S c h o o l s 2 0 1 8 - 1 9 -0.11** -64** -0.10** -57**
C h a r t e r S c h o o l s 2 0 1 7 - 1 8 0.03 15 0.02 14
C h a r t e r S c h o o l s 2 0 1 8 - 1 9 -0.02 -12 0.01 2
O p t i o n a l S c h o o l s 2 0 1 7 - 1 8 0.23** 137** 0.15* 90*
O p t i o n a l S c h o o l s 2 0 1 8 - 1 9 0.12** 70** 0.11* 66*
O t h e r D i s t r i c t S c h o o l s 2 0 1 7 - 1 8 -0.01 -5 -0.02 -13
O t h e r D i s t r i c t S c h o o l s 2 0 1 8 - 1 9 -0.03 -18 -0.01 -6
Comparison of School Sectors within Memphis
Significant at p < 0.05*
Significant at p < 0.01**
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
A S D S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 7 - 1 8 -0.08** -48** -0.06 -37
A S D S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 8 - 1 9 -0.10** -59** -0.09** -51**
C h a r t e r S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 7 - 1 8 0.03 19 0.05 27
C h a r t e r S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 8 - 1 9 0.01 5 0.01 8
O p t i o n a l S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 7 - 1 8 0.24** 141** 0.18** 103**
O p t i o n a l S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 8 - 1 9 0.15** 88** 0.12** 71**
Charter Subsector Analysis
Significant at p < 0.05*
Significant at p < 0.01**
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
M e m p h i s C M O v s . S t a t e A v e r a g e -0.03 -18 0.00 1
M e m p h i s I n d e p e n d e n t C h a r t e r s v s . S t a t e A v e r a g e -0.01 -7 0.01 4
M e m p h i s C M O v s . M e m p h i s I n d e p e n d e n t C h a r t e r s -0.02 -11 -0.01 -3
Student Subgroup Analysis> Black Students
Significant at p < 0.05*
Significant at p < 0.01**
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
Compared with Statewide Average of Black Students
M e m p h i s B l a c k S t u d e n t s O v e r a l l -0.05** -29** -0.02 -14
M e m p h i s A S D S c h o o l B l a c k S t u d e n t s -0.12** -68** -0.10** -58**
M e m p h i s C h a r t e r S c h o o l B l a c k S t u d e n t s -0.03 -21 -0.02 -9
M e m p h i s O p t i o n a l S c h o o l B l a c k S t u d e n t s 0.11** 66** 0.11* 61*
M e m p h i s O t h e r D i s t r i c t B l a c k S t u d e n t s -0.05** -29** -0.02 -11
Compared with Black Students in Other District in Memphis
M e m p h i s C h a r t e r S c h o o l B l a c k S t u d e n t s 0.01 8 0.00 1
M e m p h i s O p t i o n a l S c h o o l B l a c k S t u d e n t s 0.16** 94** 0.12* 72*
M e m p h i s A S D S c h o o l B l a c k S t u d e n t s -0.07** -40** -0.08** -48**
Student Subgroup Analysis> Hispanic Students
Significant at p < 0.05*
Significant at p < 0.01**
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
Compared with Statewide Average of Hispanic Students
M e m p h i s H i s p a n i c S t u d e n t s O v e r a l l 0.01 4 0.00 1
M e m p h i s A S D S c h o o l H i s p a n i c S t u d e n t s -0.10** -59** -0.09 -52
M e m p h i s C h a r t e r S c h o o l H i s p a n i c S t u d e n t s 0.03 17 0.12 68
M e m p h i s O p t i o n a l S c h o o l H i s p a n i c S t u d e n t s 0.03 15 0.13 75
M e m p h i s O t h e r D i s t r i c t H i s p a n i c S t u d e n t s 0.01 3 -0.02 -13
Compared with Hispanic Students in Other District in
Memphis
M e m p h i s C h a r t e r S c h o o l H i s p a n i c S t u d e n t s 0.02 14 0.14 81
M e m p h i s O p t i o n a l S c h o o l H i s p a n i c S t u d e n t s 0.02 11 0.15 88
M e m p h i s A S D S c h o o l H i s p a n i c S t u d e n t s -0.11** -62** -0.07 -39
Student Subgroup Analysis> Students in Poverty
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
Compared with Statewide Average of Students in Poverty
M e m p h i s S t u d e n t s i n P o v e r t y O v e r a l l -0.03 -16 -0.01 -6
M e m p h i s A S D S c h o o l S t u d e n t s i n P o v e r t y -0.09** -55** -0.08** -50**
M e m p h i s C h a r t e r S c h o o l S t u d e n t s i n P o v e r t y -0.01 -4 0.03 18
M e m p h i s O p t i o n a l S c h o o l S t u d e n t s i n P o v e r t y 0.16** 91** 0.14** 83**
M e m p h i s O t h e r D i s t r i c t S t u d e n t s i n P o v e r t y -0.02 -14 -0.01 -8
Compared with Students in Poverty in Other District in
Memphis
M e m p h i s C h a r t e r S c h o o l S t u d e n t s i n P o v e r t y 0.02 10 0.04 25
M e m p h i s O p t i o n a l S c h o o l S t u d e n t s i n P o v e r t y 0.18** 105** 0.16** 91**
M e m p h i s A S D S c h o o l S t u d e n t s i n P o v e r t y -0.07** -42** -0.07** -42**
Student Subgroup Analysis> ELL Students
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
Compared with Statewide Average of ELL Students
M e m p h i s E L L S t u d e n t s O v e r a l l -0.01 -5 0.01 7
M e m p h i s A S D S c h o o l E L L S t u d e n t s -0.08 -49 0.01 5
M e m p h i s C h a r t e r S c h o o l E L L S t u d e n t s -0.01 -3 0.12 70
M e m p h i s O p t i o n a l S c h o o l E L L S t u d e n t s 0.38 226 -0.19** -113**
M e m p h i s O t h e r D i s t r i c t E L L S t u d e n t s -0.01 -3 -0.01 -5
Compared with ELL Students in Other District in Memphis
M e m p h i s C h a r t e r S c h o o l E L L S t u d e n t s 0.00 0 0.13 74
M e m p h i s O p t i o n a l S c h o o l E L L S t u d e n t s 0.39 229 -0.18** -108**
M e m p h i s A S D S c h o o l E L L S t u d e n t s -0.08 -46 0.02 10
Student Subgroup Analysis> Special Ed Students
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
Compared with Statewide Average of Special Ed Students
M e m p h i s S p e c i a l E d S t u d e n t s O v e r a l l 0.01 4 0.02 10
M e m p h i s A S D S c h o o l S p e c i a l E d S t u d e n t s 0.02 10 -0.05 -28
M e m p h i s C h a r t e r S c h o o l S p e c i a l E d S t u d e n t s 0.02 14 0.03 19
M e m p h i s O p t i o n a l S c h o o l S p e c i a l E d S t u d e n t s 0.37** 219** 0.11 64
M e m p h i s O t h e r D i s t r i c t S p e c i a l E d S t u d e n t s 0.00 0 0.02 13
Compared with Special Ed Students in Other District in
Memphis
M e m p h i s C h a r t e r S c h o o l S p e c i a l E d S t u d e n t s 0.02 14 0.01 5
M e m p h i s O p t i o n a l S c h o o l S p e c i a l E d S t u d e n t s 0.37** 219** 0.09 50
M e m p h i s A S D S c h o o l S p e c i a l E d S t u d e n t s 0.02 10 -0.07 -42
Student Subgroup Analysis> Male Students
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
Compared with Statewide Average of Male Students
M e m p h i s M a l e S t u d e n t s O v e r a l l -0.02 -14 -0.01 -8
M e m p h i s A S D S c h o o l M a l e S t u d e n t s -0.12** -70** -0.09** -51**
M e m p h i s C h a r t e r S c h o o l M a l e S t u d e n t s 0.00 0 0.00 -1
M e m p h i s O p t i o n a l S c h o o l M a l e S t u d e n t s 0.14** 83** 0.13** 74**
M e m p h i s O t h e r D i s t r i c t M a l e S t u d e n t s -0.02 -13 -0.01 -6
Compared with Male Students in Other District in Memphis
M e m p h i s C h a r t e r S c h o o l M a l e S t u d e n t s 0.02 12 0.01 4
M e m p h i s O p t i o n a l S c h o o l M a l e S t u d e n t s 0.16** 96** 0.14** 80**
M e m p h i s A S D S c h o o l M a l e S t u d e n t s -0.10** -57** -0.08** -45**
Student Subgroup Analysis> Female Students
R E A D I N G M A T H
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
S t a n d a r d
D e v i a t i o n
D a y s o f
L e a r n i n g
Compared with Statewide Average of Female Students
M e m p h i s F e m a l e S t u d e n t s O v e r a l l -0.04* -24* -0.01 -6
M e m p h i s A S D S c h o o l F e m a l e S t u d e n t s -0.10** -57** -0.11** -62**
M e m p h i s C h a r t e r S c h o o l F e m a l e S t u d e n t s -0.04* -25* 0.01 7
M e m p h i s O p t i o n a l S c h o o l F e m a l e S t u d e n t s 0.10* 60* 0.10 59
M e m p h i s O t h e r D i s t r i c t F e m a l e S t u d e n t s -0.04* -22* -0.01 -5
Compared with Female Students in Other District in
Memphis
M e m p h i s C h a r t e r S c h o o l F e m a l e S t u d e n t s 0.00 -3 0.02 11
M e m p h i s O p t i o n a l S c h o o l F e m a l e S t u d e n t s 0.14** 82** 0.11 63
M e m p h i s A S D S c h o o l F e m a l e S t u d e n t s -0.06** -35** -0.10** -58**
THANK YOU

More Related Content

Similar to memphis_slide_deck_FINAL_20220429.pdf

Similar to memphis_slide_deck_FINAL_20220429.pdf (20)

Memphis, TN 2019 - City Study
Memphis, TN 2019 - City StudyMemphis, TN 2019 - City Study
Memphis, TN 2019 - City Study
 
San Antonio, TX - 2022 Charter School Study by CREDO
San Antonio, TX - 2022 Charter School Study by CREDOSan Antonio, TX - 2022 Charter School Study by CREDO
San Antonio, TX - 2022 Charter School Study by CREDO
 
Austin slide deck
Austin slide deckAustin slide deck
Austin slide deck
 
2022 Houston Chart School City Study
2022 Houston Chart School City Study2022 Houston Chart School City Study
2022 Houston Chart School City Study
 
St_Louis_slide_deck_FINAL.pdf
St_Louis_slide_deck_FINAL.pdfSt_Louis_slide_deck_FINAL.pdf
St_Louis_slide_deck_FINAL.pdf
 
batonrouge_slide_deck_FINAL_20220429.pdf
batonrouge_slide_deck_FINAL_20220429.pdfbatonrouge_slide_deck_FINAL_20220429.pdf
batonrouge_slide_deck_FINAL_20220429.pdf
 
Newark, NJ 2021
Newark, NJ 2021Newark, NJ 2021
Newark, NJ 2021
 
2022 Austin Chart School City Study
2022 Austin Chart School City Study2022 Austin Chart School City Study
2022 Austin Chart School City Study
 
DC_slide_deck_FINAL (1).pdf
DC_slide_deck_FINAL (1).pdfDC_slide_deck_FINAL (1).pdf
DC_slide_deck_FINAL (1).pdf
 
2022 Newark Chart School City Study
2022 Newark Chart School City Study2022 Newark Chart School City Study
2022 Newark Chart School City Study
 
Washington DC. City Study 2022
Washington DC. City Study 2022Washington DC. City Study 2022
Washington DC. City Study 2022
 
Denver 2019 - City Study
Denver 2019 - City StudyDenver 2019 - City Study
Denver 2019 - City Study
 
Washington DC 2019 - City Study
Washington DC 2019 - City StudyWashington DC 2019 - City Study
Washington DC 2019 - City Study
 
Baton Rouge LA 2019 - City Study
Baton Rouge LA 2019 - City StudyBaton Rouge LA 2019 - City Study
Baton Rouge LA 2019 - City Study
 
San Antonio, TX 2019 - City Study
San Antonio, TX 2019 - City StudySan Antonio, TX 2019 - City Study
San Antonio, TX 2019 - City Study
 
St. Louis, MO 2019 - City Study
St. Louis, MO 2019 - City StudySt. Louis, MO 2019 - City Study
St. Louis, MO 2019 - City Study
 
Kansas City, MO 2019 - City Study
Kansas City, MO 2019 - City StudyKansas City, MO 2019 - City Study
Kansas City, MO 2019 - City Study
 
Buffalo NY 2019 - City Study
Buffalo NY 2019 - City StudyBuffalo NY 2019 - City Study
Buffalo NY 2019 - City Study
 
Indianapolis, IN 2019 - City Study
Indianapolis, IN 2019 - City StudyIndianapolis, IN 2019 - City Study
Indianapolis, IN 2019 - City Study
 
New Orleans, LA 2019 - City Study
New Orleans, LA 2019 - City StudyNew Orleans, LA 2019 - City Study
New Orleans, LA 2019 - City Study
 

Recently uploaded

An Overview of the Odoo 17 Knowledge App
An Overview of the Odoo 17 Knowledge AppAn Overview of the Odoo 17 Knowledge App
An Overview of the Odoo 17 Knowledge AppCeline George
 
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽中 央社
 
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...Nguyen Thanh Tu Collection
 
Basic Civil Engineering notes on Transportation Engineering & Modes of Transport
Basic Civil Engineering notes on Transportation Engineering & Modes of TransportBasic Civil Engineering notes on Transportation Engineering & Modes of Transport
Basic Civil Engineering notes on Transportation Engineering & Modes of TransportDenish Jangid
 
How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17Celine George
 
OSCM Unit 2_Operations Processes & Systems
OSCM Unit 2_Operations Processes & SystemsOSCM Unit 2_Operations Processes & Systems
OSCM Unit 2_Operations Processes & SystemsSandeep D Chaudhary
 
UChicago CMSC 23320 - The Best Commit Messages of 2024
UChicago CMSC 23320 - The Best Commit Messages of 2024UChicago CMSC 23320 - The Best Commit Messages of 2024
UChicago CMSC 23320 - The Best Commit Messages of 2024Borja Sotomayor
 
ANTI PARKISON DRUGS.pptx
ANTI         PARKISON          DRUGS.pptxANTI         PARKISON          DRUGS.pptx
ANTI PARKISON DRUGS.pptxPoojaSen20
 
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes GuàrdiaPersonalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes GuàrdiaEADTU
 
How to Manage Website in Odoo 17 Studio App.pptx
How to Manage Website in Odoo 17 Studio App.pptxHow to Manage Website in Odoo 17 Studio App.pptx
How to Manage Website in Odoo 17 Studio App.pptxCeline George
 
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...Nguyen Thanh Tu Collection
 
Graduate Outcomes Presentation Slides - English (v3).pptx
Graduate Outcomes Presentation Slides - English (v3).pptxGraduate Outcomes Presentation Slides - English (v3).pptx
Graduate Outcomes Presentation Slides - English (v3).pptxneillewis46
 
Book Review of Run For Your Life Powerpoint
Book Review of Run For Your Life PowerpointBook Review of Run For Your Life Powerpoint
Book Review of Run For Your Life Powerpoint23600690
 
The Story of Village Palampur Class 9 Free Study Material PDF
The Story of Village Palampur Class 9 Free Study Material PDFThe Story of Village Palampur Class 9 Free Study Material PDF
The Story of Village Palampur Class 9 Free Study Material PDFVivekanand Anglo Vedic Academy
 
MOOD STABLIZERS DRUGS.pptx
MOOD     STABLIZERS           DRUGS.pptxMOOD     STABLIZERS           DRUGS.pptx
MOOD STABLIZERS DRUGS.pptxPoojaSen20
 
AIM of Education-Teachers Training-2024.ppt
AIM of Education-Teachers Training-2024.pptAIM of Education-Teachers Training-2024.ppt
AIM of Education-Teachers Training-2024.pptNishitharanjan Rout
 
SURVEY I created for uni project research
SURVEY I created for uni project researchSURVEY I created for uni project research
SURVEY I created for uni project researchCaitlinCummins3
 

Recently uploaded (20)

An Overview of the Odoo 17 Knowledge App
An Overview of the Odoo 17 Knowledge AppAn Overview of the Odoo 17 Knowledge App
An Overview of the Odoo 17 Knowledge App
 
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
 
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
24 ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH SỞ GIÁO DỤC HẢI DƯ...
 
Basic Civil Engineering notes on Transportation Engineering & Modes of Transport
Basic Civil Engineering notes on Transportation Engineering & Modes of TransportBasic Civil Engineering notes on Transportation Engineering & Modes of Transport
Basic Civil Engineering notes on Transportation Engineering & Modes of Transport
 
How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17
 
OSCM Unit 2_Operations Processes & Systems
OSCM Unit 2_Operations Processes & SystemsOSCM Unit 2_Operations Processes & Systems
OSCM Unit 2_Operations Processes & Systems
 
Supporting Newcomer Multilingual Learners
Supporting Newcomer  Multilingual LearnersSupporting Newcomer  Multilingual Learners
Supporting Newcomer Multilingual Learners
 
UChicago CMSC 23320 - The Best Commit Messages of 2024
UChicago CMSC 23320 - The Best Commit Messages of 2024UChicago CMSC 23320 - The Best Commit Messages of 2024
UChicago CMSC 23320 - The Best Commit Messages of 2024
 
ANTI PARKISON DRUGS.pptx
ANTI         PARKISON          DRUGS.pptxANTI         PARKISON          DRUGS.pptx
ANTI PARKISON DRUGS.pptx
 
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes GuàrdiaPersonalisation of Education by AI and Big Data - Lourdes Guàrdia
Personalisation of Education by AI and Big Data - Lourdes Guàrdia
 
How to Manage Website in Odoo 17 Studio App.pptx
How to Manage Website in Odoo 17 Studio App.pptxHow to Manage Website in Odoo 17 Studio App.pptx
How to Manage Website in Odoo 17 Studio App.pptx
 
VAMOS CUIDAR DO NOSSO PLANETA! .
VAMOS CUIDAR DO NOSSO PLANETA!                    .VAMOS CUIDAR DO NOSSO PLANETA!                    .
VAMOS CUIDAR DO NOSSO PLANETA! .
 
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
ĐỀ THAM KHẢO KÌ THI TUYỂN SINH VÀO LỚP 10 MÔN TIẾNG ANH FORM 50 CÂU TRẮC NGHI...
 
Graduate Outcomes Presentation Slides - English (v3).pptx
Graduate Outcomes Presentation Slides - English (v3).pptxGraduate Outcomes Presentation Slides - English (v3).pptx
Graduate Outcomes Presentation Slides - English (v3).pptx
 
Book Review of Run For Your Life Powerpoint
Book Review of Run For Your Life PowerpointBook Review of Run For Your Life Powerpoint
Book Review of Run For Your Life Powerpoint
 
Mattingly "AI and Prompt Design: LLMs with NER"
Mattingly "AI and Prompt Design: LLMs with NER"Mattingly "AI and Prompt Design: LLMs with NER"
Mattingly "AI and Prompt Design: LLMs with NER"
 
The Story of Village Palampur Class 9 Free Study Material PDF
The Story of Village Palampur Class 9 Free Study Material PDFThe Story of Village Palampur Class 9 Free Study Material PDF
The Story of Village Palampur Class 9 Free Study Material PDF
 
MOOD STABLIZERS DRUGS.pptx
MOOD     STABLIZERS           DRUGS.pptxMOOD     STABLIZERS           DRUGS.pptx
MOOD STABLIZERS DRUGS.pptx
 
AIM of Education-Teachers Training-2024.ppt
AIM of Education-Teachers Training-2024.pptAIM of Education-Teachers Training-2024.ppt
AIM of Education-Teachers Training-2024.ppt
 
SURVEY I created for uni project research
SURVEY I created for uni project researchSURVEY I created for uni project research
SURVEY I created for uni project research
 

memphis_slide_deck_FINAL_20220429.pdf

  • 2. Table of Contents 0 1 R E PORT OVE R VIE W • About The City Studies Project • Sectors of Schools • Research Question and Analyses • Measure of Academic Performance Student Subgroup Analysis Summary of Findings 0 3 APPE N DIX E S • Acknowledgments • Types of Charter Schools • Methods • Days of Learning 0 2 R E SE AR CH F IN DIN G S • Reading & math Overall Memphis Results Sector Analysis • vs. state & comparison within Memphis Charter Subsector Analysis Research Findings Cont’d. Black Students • vs. state & comparison within Memphis Hispanic Students • vs. state & comparison within Memphis Students in Poverty • vs. state & comparison within Memphis ELL Students • vs. state & comparison within Memphis Special Ed Students • vs. state & comparison within Memphis Male Students • vs. state & comparison within Memphis Female Students • vs. state & comparison within Memphis • vs. state • comparison within Memphis
  • 4. About The City Studies Project Cohort 1 Cohort 2 The City Studies project examines the performance of schools in select U.S. cities, including Memphis. We study the academic progress of students as the measure of school performance.
  • 5. O O C C CHARTER SCHOOLS Public schools operated independently from the traditional school district, with autonomy in adapting school designs and held accountable for education results. Charter Management Organizations (CMOs) Organizations holding the charter and overseeing the operation of at least three charter schools. Independent Charter Schools Organizations holding the charter and overseeing the operation of a single or two charter schools. ACHIEVEMENT SCHOOL DISTRICT (ASD) SCHOOLS Schools needing improvement that are directly run by the district’s staff or operated by charter management organizations (CMOs) authorized by the district. OTHER DISTRICT-RUN SCHOOLS Public schools not belonging to any of above types. C Sectors of Schools COMMUNITIES MAY HAVE UP TO FOUR SECTORS OF SCHOOLS OPTIONAL SCHOOLS District-run schools that screen all the students with academic criteria.
  • 6. Research Question and Analyses IN THIS REPORT WE EXAMINE ACADEMIC PERFORMANCE IN MEMPHIS USING DATA FROM THE SCHOOL YEARS 2016-17 THROUGH 2018-19. THERE ARE THREE LEVELS OF ANALYSIS. Overall performance in Memphis schools over two years. • The performance of Memphis students is benchmarked against the state average performance, accounting for student characteristics. • The performance of ASD school students, performance of charter school students, performance of optional school students within Memphis are then compared to that of similar traditional public school (other district school) students within Memphis. Performance for ASD schools, charter schools, optional schools, and the rest of Memphis public schools over two years Performance in the 2018- 2019 school year by school type, race, poverty status, English language learner (ELL) status, special education status and gender. WE MAKE TWO SETS OF COMPARISONS. 01 02 03
  • 7. Achievement scores capture what a student knows at a point in time. They are influenced by students’ prior conditions in addition to schools’ contributions. Growth scores indicate how much progress a student makes from one year to the next. Growth scores allow us to zero in on the contributions of schools separately from other factors that affect point-in-time scores. ACHIEVEMENT VS. GROWTH We analyze student growth in standard deviation units so that the results can be assessed for statistical differences. In the following tables of findings, we transform growth from standard deviation units into days of learning based on a typical 180-day school year. IN THIS STUDY WE MEASURE ACADEMIC PERFORMANCE AS HOW MUCH GROWTH STUDENTS MAKE FROM ONE YEAR TO THE NEXT Measure of Academic Performance
  • 9. -250 -200 -150 -100 -50 0 50 100 150 200 250 SY 2017-2018 SY 2018-2019 Growth (in Days of Learning) significantly different at p< 0.05 reading math Research Findings > Overall Memphis Results > Reading & Math Average One-Year Learning Gains for All Memphis Students Compared to the State Average Learning Gains, by Year and Subject
  • 10. -250 -200 -150 -100 -50 0 50 100 150 200 250 SY 2017-2018 SY 2018-2019 Growth (in Days of Learning) Learning Gains in Reading for Students in Memphis Charter Schools, Memphis ASD Schools, Memphis Optional Schools and Memphis District Compared to the State Average Learning Gains by Year significantly different at p< 0.05 charter innovation Research Findings > Sector Analysis > Reading VS. STATE & COMPARISON WITHIN MEMPHIS charter innovation district optional Reading Charter vs. District Optional vs. District ASD vs. District Charter vs. Optional Charter vs. ASD '17-'18 '18-'19 Tests of Differences
  • 11. -250 -200 -150 -100 -50 0 50 100 150 200 250 SY 2017-2018 SY 2018-2019 Growth (in Days of Learning) significantly different at p< 0.05 charter innovation district Research Findings > Sector Analysis > Math VS. STATE & COMPARISON WITHIN MEMPHIS Learning Gains in Math for Students in Memphis Charter Schools, Memphis ASD Schools, Memphis Optional Schools and Memphis District Compared to the State Average Learning Gains by Year optional Math Charter vs. District Optional vs. District ASD vs. District Charter vs. Optional Charter vs. ASD Tests of Differences '17-'18 '18-'19
  • 12. -250 -200 -150 -100 -50 0 50 100 150 200 250 CMO Charter Schools Independent Charter Schools Growth (in Days of Learning) Relative Learning Gains for Students in Memphis Network-Affiliated Charter Schools and Independent Memphis Charter Schools Compared to the Average Learning Gains for All Student in the State, by Subject significantly different at p< 0.05 reading math Research Findings > Charter Subsector Analysis > vs. state & comparison within Memphis Reading CMOs vs Independent Charter Schools Math CMOs vs Independent Charter Schools sig Tests of Differences
  • 13. -600 -500 -400 -300 -200 -100 0 100 200 300 400 500 Growth (in Days of Learning) Charter District Optional ASD Research Findings > School-Level Performance by Sector > Reading
  • 14. -600 -500 -400 -300 -200 -100 0 100 200 300 400 500 Growth (in Days of Learning) Charter District Optional ASD Research Findings > School-Level Performance by Sector >Math
  • 15. -150 -100 -50 0 50 100 150 200 250 Memphis Black Students Growth (in Days of Learning) Learning Gains for All Memphis Black Students Compared to the Average Black Learning Gains of Black Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Black Students ALL VS. STATE
  • 16. -150 -100 -50 0 50 100 150 200 250 Memphis Charter Black Students Memphis Optional Black Students Memphis District Black Students Memphis ASD Black Students Growth (in Days of Learning) Learning Gains for Black Charter School Students, Black Innovation School Students, and Black District Students in Memphis Compared to the Average Learning Gains of Black Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Black Students VS. STATE BY SECTOR Reading Charter Black vs. District Black Optional Black vs. District Black ASD Black vs. District Black Math Charter Black vs. District Black Optional Black vs. District Black ASD Black vs. District Black Tests of Differences sig
  • 17. -150 -100 -50 0 50 100 150 200 250 Memphis Hispanic Students Growth (in Days of Learning) Learning Gains for All Memphis Hispanic Students Compared to the Hispanic Average Learning Gains of Hispanic Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Hispanic Students ALL VS. STATE
  • 18. -150 -100 -50 0 50 100 150 200 250 Memphis Charter Hispanic Students Memphis Optional Hispanic Students Memphis District Hispanic Students Memphis ASD Hispanic Students Growth (in Days of Learning) Learning Gains for Hispanic Charter School Students, Hispanic Innovation School Students, and Hispanic District Students in Memphis Compared to the Average Learning Gains of Hispanic Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Hispanic Students VS. STATE BY SECTOR Reading Charter Hispanic vs. District Hispanic Optional Hispanic vs. District Hispanic ASD Hispanic vs. District Hispanic Math Charter Hispanic vs. District Hispanic Optional Hispanic vs. District Hispanic ASD Hispanic vs. District Hispanic Tests of Differences sig
  • 19. -150 -100 -50 0 50 100 150 200 250 Memphis Students in Poverty Growth (in Days of Learning) Learning Gains for All Memphis Students in Poverty Compared to the Poverty Average Learning Gains of Students in Poverty Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Students in Poverty ALL VS. STATE
  • 20. -150 -100 -50 0 50 100 150 200 250 Memphis Charter Students in Poverty Memphis Optional Students in Poverty Memphis District Students in Poverty Memphis ASD Students in Poverty Growth (in Days of Learning) Learning Gains for Memphis Charter School Students in Poverty, Memphis Innovation School Students in Poverty, and Memphis District Students in Poverty Compared to the Average Learning Gains of Students in Poverty Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Students in Poverty VS. STATE BY SECTOR Reading Charter Poverty vs. District Poverty Optional Poverty vs. District Poverty ASD Poverty vs. District Poverty Math Charter Poverty vs. District Poverty Optional Poverty vs. District Poverty ASD Poverty vs. District Poverty Tests of Differences sig
  • 21. -150 -100 -50 0 50 100 150 200 250 Memphis ELL Students Growth (in Days of Learning) Learning Gains for All ELL Students in Memphis Compared to the Average ELL Learning Gains of ELL Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > ELL Students ALL VS. STATE
  • 22. -150 -100 -50 0 50 100 150 200 250 Memphis Charter ELL Students Memphis Optional ELL Students Memphis District ELL Students Memphis ASD ELL Students Growth (in Days of Learning) Learning Gains for ELL Students in Memphis Charter Schools, ELL Students in Memphis Innovation Schools, and ELL Students in Memphis District Compared to the Average Learning Gains of ELL Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > ELL Students VS. STATE BY SECTOR Reading Charter ELL vs. District ELL Optional ELL vs. District ELL ASD ELL vs. District ELL Math Charter ELL vs. District ELL Optional ELL vs. District ELL ASD ELL vs. District ELL Tests of Differences sig
  • 23. Learning Gains for All Memphis Students in Special Education Compared to the Average Learning Gains of Students in Special Education Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Special Ed Students ALL VS. STATE -150 -100 -50 0 50 100 150 200 250 Memphis Special Ed Students Growth (in Days of Learning)
  • 24. -150 -100 -50 0 50 100 150 200 250 Memphis Charter Students in Special Ed. Memphis Optional Students in Special Ed. Memphis District Students in Special Ed. Memphis ASD Students in Special Ed. Growth (in Days of Learning) Learning Gains for Memphis Charter School Students in Special Ed., Memphis Innovation School Students in Special Ed., and Memphis District Students in Special Ed. Compared to the Average Learning Gains of Students in Special Ed. Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Special Ed Students VS. STATE BY SECTOR Reading Charter Sped vs. District Sped Optional Sped vs. District Sped ASD Sped vs. District Sped Math Charter Sped vs. District Sped Optional Sped vs. District Sped ASD Sped vs. District Sped Tests of Differences sig
  • 25. -150 -100 -50 0 50 100 150 200 250 Memphis Male Students Growth (in Days of Learning) Learning Gains for All Memphis Male Students Compared to the Average Male Learning Gains of Male Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Male Students ALL VS. STATE
  • 26. -150 -100 -50 0 50 100 150 200 250 Memphis Charter Male Students Memphis Optional Male Students Memphis District Male Students Memphis ASD Male Students Growth (in Days of Learning) Learning Gains for Male Charter School Students, Male Innovation School Students, and Male District Students in Memphis compared to the Average Learning Gains of Male Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Male Students VS. STATE BY SECTOR Reading Charter Male vs. District Male Magnet Male vs. District Male Innovation Male vs. District Male Math Charter Male vs. District Male Magnet Male vs. District Male Innovation Male vs. District Male Tests of Differences sig Reading Charter Male vs. District Male Optional Male vs. District Male ASD Male vs. District Male Math Charter Male vs. District Male Optional Male vs. District Male ASD Male vs. District Male Tests of Differences sig
  • 27. -150 -100 -50 0 50 100 150 200 250 Memphis Female Students Growth (in Days of Learning) Learning Gains for All Memphis Female Students Compared to the Female Average Learning Gains of Female Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Female Students ALL VS. STATE
  • 28. -150 -100 -50 0 50 100 150 200 250 Memphis Charter Female Students Memphis Optional Female Students Memphis District Female Students Memphis ASD Female Students Growth (in Days of Learning) Learning Gains for Female Charter School Students, Female Innovation School Students, and Female District Students in Memphis Compared to the Average Learning Gains of Female Students Statewide, by Subject significantly different at p< 0.05 reading math Research Findings > Student Subgroup Analysis > Female Students VS. STATE BY SECTOR Reading Charter Female vs. District Female Optional Female vs. District Female ASD Female vs. District Female Math Charter Female vs. District Female Optional Female vs. District Female ASD Female vs. District Female Tests of Differences sig
  • 29. Summary of Findings The summary of the findings from the analysis of Memphis schools is presented here.
  • 31. Acknowledgments Tennessee Department of Education and Tennessee Education Research Alliance provided supports in the acquisition of student-level data. Shelby County Schools assisted CREDO with verifying the list of public schools in Memphis.
  • 32. Types of Charter Schools • With more schools and students than a single charter school, CMOs have some operational advantages in their ability to spread administrative fixed costs, thus providing the possibility of greater efficiency. In addition, CMOs may be able to support additional programs and more robust staffing. • Whether CMOs lead to better student outcomes is a matter of interest across the country. OUR ANALYSES OF MEMPHIS CHARTER SCHOOLS INCLUDE A BREAKOUT OF CMOS AND INDEPENDENT CHARTERS. There are two types of charter schools. CHARTER MANAGEMENT ORGANIZATIONS (CMOS) Organizations holding the charter and overseeing the operation of at least three charter schools. INDEPENDENT CHARTER SCHOOLS Organization holding the charter and overseeing the operation of a single charter school. It may run the school directly or contract with an organization which provides services to one or two charter schools.
  • 33. Methods The annual academic growth of students in Memphis from 2016-17 to 2018-19, overall and by sector, is benchmarked to the state average growth, accounting for student characteristics. We also explore how one-year growth of Memphis students for the period ending in Spring 2019 differs by school type, race, poverty status, English language learner status, special education status, and gender.
  • 34. Days of Learning While these tools create precise and reliable answers, they are presented in technical terms that are not user-friendly to a general audience. To translate the technical results into terms that are accessible to non- technical audiences, CREDO developed Days of Learning. CREDO USES ADVANCED TECHNOLOGY AND SOPHISTICATED STATISTICAL TOOLS TO MEASURE STUDENTS, SCHOOLS AND THE EDUCATION LANDSCAPE. Think about the students in your state’s public schools. For many of their years of schooling, they take achievement tests to measure what they know at the end of the school year. We can identify the average score for each test each year. Imagine a student who scores exactly at the average in one year, say 4th grade, and then in the following year, scores exactly at the average again on the 5th- grade test. The amount of year-to-year learning for that student show us what the average learning is for all the students who took both tests. We do that calculation for every grade the state tests: 4th to 5th, 5th to 6th, and so on. CREDO uses those annual measures of average learning to represent a typical year of learning, and equates that to a typical 180-day school year. We say that the student in our example has gained 180 days of learning. If a student makes more progress than the average student, we take the amount of extra achievement and translate it into 180-days of learning plus “X” extra days. We are creating a measure of student learning as if the student went to school for 180 days plus X days. The size of “X” depends on how much more the student learns than the average student — if it’s a lot more, then “X” will be a large number, and if it’s a small amount more, “X” will be a small number. The same is true for students who do not learn as much as the average student. Instead of adding to the 180-days-of-learning average, we subtract from that base to reflect the smaller-than-average advances that those students realize. In these cases, the difference leads to numbers such a “165 days of learning” or “152 days of learning”. Against the average standard of 180 days, these smaller days show that students learned as if they had only attended school for 180 days minus X days during the school year. 01 02 03 04 05 06 3rd Grade 4th Grade = 180 days More than 180 days Less than 180 days Student A Student A
  • 35. Overall Memphis Results Significant at p < 0.05* Significant at p < 0.01** R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g M e m p h i s O v e r a l l 2 0 1 7 - 1 8 0.00 -3 -0.02 -10 M e m p h i s O v e r a l l 2 0 1 8 - 1 9 -0.03* -19* -0.01 -7
  • 36. Memphis School Sectors Compared to State Average Significant at p < 0.05* Significant at p < 0.01** R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g A S D S c h o o l s 2 0 1 7 - 1 8 -0.09** -52** -0.08* -50* A S D S c h o o l s 2 0 1 8 - 1 9 -0.11** -64** -0.10** -57** C h a r t e r S c h o o l s 2 0 1 7 - 1 8 0.03 15 0.02 14 C h a r t e r S c h o o l s 2 0 1 8 - 1 9 -0.02 -12 0.01 2 O p t i o n a l S c h o o l s 2 0 1 7 - 1 8 0.23** 137** 0.15* 90* O p t i o n a l S c h o o l s 2 0 1 8 - 1 9 0.12** 70** 0.11* 66* O t h e r D i s t r i c t S c h o o l s 2 0 1 7 - 1 8 -0.01 -5 -0.02 -13 O t h e r D i s t r i c t S c h o o l s 2 0 1 8 - 1 9 -0.03 -18 -0.01 -6
  • 37. Comparison of School Sectors within Memphis Significant at p < 0.05* Significant at p < 0.01** R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g A S D S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 7 - 1 8 -0.08** -48** -0.06 -37 A S D S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 8 - 1 9 -0.10** -59** -0.09** -51** C h a r t e r S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 7 - 1 8 0.03 19 0.05 27 C h a r t e r S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 8 - 1 9 0.01 5 0.01 8 O p t i o n a l S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 7 - 1 8 0.24** 141** 0.18** 103** O p t i o n a l S c h o o l s v s . O t h e r D i s t r i c t 2 0 1 8 - 1 9 0.15** 88** 0.12** 71**
  • 38. Charter Subsector Analysis Significant at p < 0.05* Significant at p < 0.01** R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g M e m p h i s C M O v s . S t a t e A v e r a g e -0.03 -18 0.00 1 M e m p h i s I n d e p e n d e n t C h a r t e r s v s . S t a t e A v e r a g e -0.01 -7 0.01 4 M e m p h i s C M O v s . M e m p h i s I n d e p e n d e n t C h a r t e r s -0.02 -11 -0.01 -3
  • 39. Student Subgroup Analysis> Black Students Significant at p < 0.05* Significant at p < 0.01** R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g Compared with Statewide Average of Black Students M e m p h i s B l a c k S t u d e n t s O v e r a l l -0.05** -29** -0.02 -14 M e m p h i s A S D S c h o o l B l a c k S t u d e n t s -0.12** -68** -0.10** -58** M e m p h i s C h a r t e r S c h o o l B l a c k S t u d e n t s -0.03 -21 -0.02 -9 M e m p h i s O p t i o n a l S c h o o l B l a c k S t u d e n t s 0.11** 66** 0.11* 61* M e m p h i s O t h e r D i s t r i c t B l a c k S t u d e n t s -0.05** -29** -0.02 -11 Compared with Black Students in Other District in Memphis M e m p h i s C h a r t e r S c h o o l B l a c k S t u d e n t s 0.01 8 0.00 1 M e m p h i s O p t i o n a l S c h o o l B l a c k S t u d e n t s 0.16** 94** 0.12* 72* M e m p h i s A S D S c h o o l B l a c k S t u d e n t s -0.07** -40** -0.08** -48**
  • 40. Student Subgroup Analysis> Hispanic Students Significant at p < 0.05* Significant at p < 0.01** R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g Compared with Statewide Average of Hispanic Students M e m p h i s H i s p a n i c S t u d e n t s O v e r a l l 0.01 4 0.00 1 M e m p h i s A S D S c h o o l H i s p a n i c S t u d e n t s -0.10** -59** -0.09 -52 M e m p h i s C h a r t e r S c h o o l H i s p a n i c S t u d e n t s 0.03 17 0.12 68 M e m p h i s O p t i o n a l S c h o o l H i s p a n i c S t u d e n t s 0.03 15 0.13 75 M e m p h i s O t h e r D i s t r i c t H i s p a n i c S t u d e n t s 0.01 3 -0.02 -13 Compared with Hispanic Students in Other District in Memphis M e m p h i s C h a r t e r S c h o o l H i s p a n i c S t u d e n t s 0.02 14 0.14 81 M e m p h i s O p t i o n a l S c h o o l H i s p a n i c S t u d e n t s 0.02 11 0.15 88 M e m p h i s A S D S c h o o l H i s p a n i c S t u d e n t s -0.11** -62** -0.07 -39
  • 41. Student Subgroup Analysis> Students in Poverty R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g Compared with Statewide Average of Students in Poverty M e m p h i s S t u d e n t s i n P o v e r t y O v e r a l l -0.03 -16 -0.01 -6 M e m p h i s A S D S c h o o l S t u d e n t s i n P o v e r t y -0.09** -55** -0.08** -50** M e m p h i s C h a r t e r S c h o o l S t u d e n t s i n P o v e r t y -0.01 -4 0.03 18 M e m p h i s O p t i o n a l S c h o o l S t u d e n t s i n P o v e r t y 0.16** 91** 0.14** 83** M e m p h i s O t h e r D i s t r i c t S t u d e n t s i n P o v e r t y -0.02 -14 -0.01 -8 Compared with Students in Poverty in Other District in Memphis M e m p h i s C h a r t e r S c h o o l S t u d e n t s i n P o v e r t y 0.02 10 0.04 25 M e m p h i s O p t i o n a l S c h o o l S t u d e n t s i n P o v e r t y 0.18** 105** 0.16** 91** M e m p h i s A S D S c h o o l S t u d e n t s i n P o v e r t y -0.07** -42** -0.07** -42**
  • 42. Student Subgroup Analysis> ELL Students R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g Compared with Statewide Average of ELL Students M e m p h i s E L L S t u d e n t s O v e r a l l -0.01 -5 0.01 7 M e m p h i s A S D S c h o o l E L L S t u d e n t s -0.08 -49 0.01 5 M e m p h i s C h a r t e r S c h o o l E L L S t u d e n t s -0.01 -3 0.12 70 M e m p h i s O p t i o n a l S c h o o l E L L S t u d e n t s 0.38 226 -0.19** -113** M e m p h i s O t h e r D i s t r i c t E L L S t u d e n t s -0.01 -3 -0.01 -5 Compared with ELL Students in Other District in Memphis M e m p h i s C h a r t e r S c h o o l E L L S t u d e n t s 0.00 0 0.13 74 M e m p h i s O p t i o n a l S c h o o l E L L S t u d e n t s 0.39 229 -0.18** -108** M e m p h i s A S D S c h o o l E L L S t u d e n t s -0.08 -46 0.02 10
  • 43. Student Subgroup Analysis> Special Ed Students R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g Compared with Statewide Average of Special Ed Students M e m p h i s S p e c i a l E d S t u d e n t s O v e r a l l 0.01 4 0.02 10 M e m p h i s A S D S c h o o l S p e c i a l E d S t u d e n t s 0.02 10 -0.05 -28 M e m p h i s C h a r t e r S c h o o l S p e c i a l E d S t u d e n t s 0.02 14 0.03 19 M e m p h i s O p t i o n a l S c h o o l S p e c i a l E d S t u d e n t s 0.37** 219** 0.11 64 M e m p h i s O t h e r D i s t r i c t S p e c i a l E d S t u d e n t s 0.00 0 0.02 13 Compared with Special Ed Students in Other District in Memphis M e m p h i s C h a r t e r S c h o o l S p e c i a l E d S t u d e n t s 0.02 14 0.01 5 M e m p h i s O p t i o n a l S c h o o l S p e c i a l E d S t u d e n t s 0.37** 219** 0.09 50 M e m p h i s A S D S c h o o l S p e c i a l E d S t u d e n t s 0.02 10 -0.07 -42
  • 44. Student Subgroup Analysis> Male Students R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g Compared with Statewide Average of Male Students M e m p h i s M a l e S t u d e n t s O v e r a l l -0.02 -14 -0.01 -8 M e m p h i s A S D S c h o o l M a l e S t u d e n t s -0.12** -70** -0.09** -51** M e m p h i s C h a r t e r S c h o o l M a l e S t u d e n t s 0.00 0 0.00 -1 M e m p h i s O p t i o n a l S c h o o l M a l e S t u d e n t s 0.14** 83** 0.13** 74** M e m p h i s O t h e r D i s t r i c t M a l e S t u d e n t s -0.02 -13 -0.01 -6 Compared with Male Students in Other District in Memphis M e m p h i s C h a r t e r S c h o o l M a l e S t u d e n t s 0.02 12 0.01 4 M e m p h i s O p t i o n a l S c h o o l M a l e S t u d e n t s 0.16** 96** 0.14** 80** M e m p h i s A S D S c h o o l M a l e S t u d e n t s -0.10** -57** -0.08** -45**
  • 45. Student Subgroup Analysis> Female Students R E A D I N G M A T H S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g S t a n d a r d D e v i a t i o n D a y s o f L e a r n i n g Compared with Statewide Average of Female Students M e m p h i s F e m a l e S t u d e n t s O v e r a l l -0.04* -24* -0.01 -6 M e m p h i s A S D S c h o o l F e m a l e S t u d e n t s -0.10** -57** -0.11** -62** M e m p h i s C h a r t e r S c h o o l F e m a l e S t u d e n t s -0.04* -25* 0.01 7 M e m p h i s O p t i o n a l S c h o o l F e m a l e S t u d e n t s 0.10* 60* 0.10 59 M e m p h i s O t h e r D i s t r i c t F e m a l e S t u d e n t s -0.04* -22* -0.01 -5 Compared with Female Students in Other District in Memphis M e m p h i s C h a r t e r S c h o o l F e m a l e S t u d e n t s 0.00 -3 0.02 11 M e m p h i s O p t i o n a l S c h o o l F e m a l e S t u d e n t s 0.14** 82** 0.11 63 M e m p h i s A S D S c h o o l F e m a l e S t u d e n t s -0.06** -35** -0.10** -58**