The document investigates the impact of school principals on public sector productivity, particularly in relation to student outcomes and teacher turnover in high poverty schools. It highlights the methodological challenges in measuring principal effectiveness and argues that principal quality significantly influences school performance, with a one standard deviation increase in principal quality correlating with improvements in school average achievement. The findings suggest that less effective principals are more likely to leave their positions, often transitioning to other schools, especially those with higher poverty rates.

1. Estimating the Effect of Leaders on
Public Sector Productivity: The Case
of School Principals
Eric Hanushek, Steven Rivkin, and Greg Branch
January 2013

2. Questions
• Does the quality of leadership explain a substantial share
of the variation in organization outcomes?
• Does the variation in principal effectiveness differ by the
share of low income students in a school?
• Is the pattern of teacher turnover consistent with the
notion that raising the quality of teachers constitutes an
important mechanism through which principals influence
school quality?
• Are “effective” principals more likely to leave high
poverty schools?

3. Methodological challenges
• Difficult to separate contributions of
leaders from other factors
– Observed characteristics explain little of the
variation in student performance
– Typically behaviors cannot be observed or
directly related to outcomes
– Semi-parametric analyses infer effectiveness
from contributions to student outcomes
• Unlike the case for teachers, principal actions often
affect quality of school in future periods

4. Growing Literature on Principals
• Early work focused on observed
characteristics
– Clark, Martorell and Rockoff also consider
experience
• Other work builds on the panel data
methods used by Bertrand and Schoar
– raise methodological concerns
• Grissom and Loeb; Miller (2009)

5. Direct estimation of Variance in
Principal Effectiveness
• Builds on Rivkin, Hanushek and Kain
(2005) research on teacher quality
– Variation in achievement increases with a
principal change
• Coelli and Green (2012) use this approach
for Canada

6. Our Approaches
• Use value-added model with school fixed
effects to estimate principal fixed effects
– Compute variance of principal effectiveness
• Infer variance in principal effectiveness
from relationship between year-to-year
fluctuations in school value-added and
principal turnover
– Carefully examining sensitivity to timing

7. • Examine relationship between estimates
of principal quality and changes in teacher
quality during a principal’s tenure
• Describe principal quality differences by
transition status
• Consider differences by poverty share
throughout analysis

8. UTD Texas Schools Project
• Stacked panels of students and staff
• Annual testing
• Student demographic characteristics
– Divide schools by student poverty rate
• Information on staff
– Role
– Experience
– school
• Can follow principals and students who switch
schools and roles within Texas public schools

9. Estimation of Variation in
Principal Quality-Broad Issues
• Non-random selection of principals and students
into schools complicates analysis
• Tenure-quality relationship complex
– Length of tenure unlikely to be monotonically related
to effectiveness
• Principals likely learn from experience
– Skills
– Behavior rewarded in school and district
• Principal effects on school quality likely grow in
magnitude over time
• May be positive or negative

10. First empirical approach
• Principal by spell fixed effects based
on first three years at a school
– Regress math score on lagged math
score, student demographic variables,
and grade by year fixed effects using
aggregate data
– bias potentially introduced by
unobserved school factors

11. Alternative Approach
• Ignore issue of tenure and use all
spells
• Control for school fixed effects
• Potential bias if there are time-varying
school factors not accounted for
• Estimate of variance includes
differences due to tenure

12. Test Measurement Issues
• Random error inflates estimates of
variation in principal quality
– Use Bayesian shrinkage estimator to
mitigate effects of random error
– Unlike the case with the estimation of
teacher quality, it is not a serious
problem given adequacy of sample sizes
even in small schools

13. • Tests focus on basic skills, so initial
achievement differences may
influence translation of principal quality
into test score growth
• Create Z scores and re-weight
observations such that average
achievement in all schools aggregates
over the same test distribution in terms
of the share of students in each of ten
deciles of the pre-test distribution

14. Table 3. Distribution of Principal by Spell Fixed
Effects by Low Income Share
Share Standard 10th 25th 75th 90th
deviation
low inc
quartile
all -0.29 -0.15 0.11 0.22
0.21
lowest 0.16 -0.18 -0.06 0.13 0.22
2nd 0.18 -0.24 -0.14 0.09 0.19
3rd 0.21 -0.30 -0.16 0.10 0.21
highest 0.26 -0.38 -0.24 0.11 0.29

15. Sensitivity checks
• Shrink and reweight
• School fixed effects included in
specification estimated over sample of all
schools with multiple principals during
period

16. Alternative, Test-Measurement Error Adjusted
Estimates of the Variance in Principal
Effectiveness
Adjustment Neither Shrunk Reweighted Shrunk and
Shrunk nor Reweighted
Reweighted
Standard 0.207 0.200 0.270 0.241
deviation

18. Deficiencies of fixed effects
• Unobservables, even if orthogonal to
principal quality, inflate variance estimate
• These include changes over time in student
cohort quality and district curricula
• Not accounted for with shrinkage

19. Principal turnover based
variance estimates
• Derive variance estimates from
relationship between year-to-year changes
in school average achievement and
principal turnover
– If principal quality matters, changes should be
larger in years in which there is a change in
principal
– Builds on Rivkin et al (2005) estimates of
variance in teacher effectiveness

20. • Variance in principal effectiveness equals
additional year-to-year variation in
transition years over non-transition years
– Fluctuations between non-transition years
provide valid counterfactual for what would
have taken place in transition years in the
absence of a change in leadership
– Not valid if there is additional turbulence
during transition years (e.g. Ashenfelter dip)

21. • If it is actually caused by principal quality,
differences in school quality should be
larger in non-adjacent years due to an
increase over time in principal effects
– Compare non-adjacent years around
transitions in order to investigate source of
additional variation

22. specification
principal quality (θ) in cohort y,
quality of other school factors not under the control
of the principal (γ)

23. Taking Expectation
E (∆ Asy − ∆ Asy ' ) 2 = 2(σ θ2s − σ θ2yθ y ' ) + 2(σ θ2s − σ θ2yθ y ' ) + E (es )
s s s s
Assume cov(principal quality) = var(principal quality) if
principal same
Assume cov(principal quality) = 0 if principal different

24. • Regress squared year-to-year difference
in school average test score gains on
indicator for principal change
• Assumptions to identify within school
variance in principal quality from turnover
coefficient
– Principal turnover orthogonal to other
unobserved changes that affect achievement
– Schools draw principals from common
distributions during this period

25. Within School Covariance
• Covariance between principal quality in
adjacent years with same principal in both
years equals variance in principal quality
• Covariance between principal quality in
adjacent years equals zero in schools that
change principals
• Coefficient on the principal turnover
indicator equals 2 times variance in
principal quality

26. Sensitivity checks
• Add squared differences in demographic
characteristics in some specifications
• Use non-adjacent years in some
specifications

27. Results for entire sample
Timing of Comparison Adjacent year one year in
between
Student Demographic
and Mobility controls no yes no yes
Different Principal 0.0052 0.0048 0.0058 0.0056
Coefficient (3.41) (3.16) (4.35) (4.28)
estimated standard deviation 0.051 0.049 0.054 0.053
of principal quality
(square root of 0.5*coefficient)

28. Estimated Standard Deviation
by Poverty Quartile
Quartile Lowest 2nd 3rd highest
Adjacent 0.029 0.037 0.049* 0.067
years
Non- 0.027 0.035 0.057* 0.064
adjacent
years

29. Principal quality and teacher
turnover
• Principal may have limited control over
entrants
– Job security an issue, but can still exert
influence over who remains
• Desirability of school for high quality teachers
• Decision to move out lower performers
• Focus on effectiveness of exiting teachers
and rate of turnover

30. Figure 1. Teacher Transitions by Principal
Effectiveness and School Poverty Rate
Lowest Quartile Disadvantaged 2nd Quartile Disadvantaged
Bottom Bottom
2nd 2nd
3rd 3rd
Top Top
3rd Quartile Disadvantaged Highest Quartile Disadvantaged
Bottom Bottom
2nd 2nd
3rd 3rd
Top Top
0 .1 .2 .3 0 .1 .2 .3
Quartiles Principle Effectiveness
Change School Change District
Exit Sample

31. Estimation
• Argument is that better principals are
more likely to “dismiss” least effective
teachers
• Data does not link students and teachers
– Focus on differences in grade average value-
added within schools
• Grade with lower mean value-added is
more likely to have a teacher below the
“dismissal” threshold

32. Campus by year fixed effect
regressions
• Regress share of teachers that exit grade g in
school s in year y on controls and grade average
value added interacted with principal quality
quartile indicators
• Control for
– student demographics
– grade by year fixed effects
– School by year fixed effects
• Potential problem of a mechanical relationship
– In future plan to sever time periods
• Measure quality with data in 2nd year of spell
• Examine link with teacher turnover in subsequent years

33. Table 8. Coefficients on Principal Quality Quartile-Grade Average
Value-Added Interactions Using First Three Years Sample by
School Poverty
Poverty Quartiles all highest
Grade average gain*2nd quartile
-0.018 -0.065
principal quality (0.89) (1.79)
Grade average gain*3rd quartile
-0.029 -0.025
principal quality (1.35) (0.65)
Grade average gain*4th quartile
-0.079 -0.102
principal quality (3.68) (3.16)

34. Principal Transitions and Value
Added
• Transitions categorized by new role and destination
• New role
– Principal
– Other position in school
– Central office administrator
• Destination
– same school
– New school-same district
– Central office-same district
– New school-New district
– Central office-new district
– Exit Texas public schools

35. Probability Principal Remains in Same Position following 3rd
Year in a School, by Quartile of Estimated Quality and
School Poverty Rate (<25 yrs ex)
Principal Quality
Quartile Lowest 2nd 3rd highest
School Poverty
Quartile
Lowest 59% 68% 73% 76%
2nd 52% 70% 81% 72%
3rd 44% 55% 64% 58%
Highest 63% 73% 72% 67%

36. Probability Principal with Fewer than 25 Years of Experience
Becomes Principal in Different School following 3rd Year in a
School, by Quartile of Estimated Quality and School Poverty
Rate (total probability of changing position)
Principal Quality Quartile lowest 2nd 3rd highest
School Poverty
Quartile
lowest 7%(41) 6%(32) 8%(27) 9%(24)
2nd 5%(48) 8%(30) 3%(19) 12%(28)
3rd 12%(56) 11%(45) 11%(36) 15%(42)
highest 12%(37) 12%(27) 10%(28) 9%(33)

37. Future Work
• Estimate very flexible model with school
by year fixed effects
• Use these estimates to examine whether
variance in estimates of principal quality
rises with tenure in a school
• Account for Ashenfelter dip in direct
estimates of variance in principal quality
• Modify teacher turnover analysis

38. Summary
• Purposeful sorting and unobserved factors
complicate estimates of leadership quality
distribution
• We find substantial variation in estimates of
principal quality
– A one standard deviation increase in principal quality
would increase school average achievement by
roughly 0.05 standard deviations (roughly half as
much as a one std dev increase in teacher quality
• Least effective principals least likely to remain in
a school
– Often transition to other schools, particularly from a
high poverty school

39. • Details
– Turnover based estimates ignore any between school
variation in principal quality
– Find a higher quality variance in high poverty schools
– Direct estimates of principal quality appear to
overstate variance, even in specifications that include
school fixed effects
• Evidence is consistent with notion that the
management of teacher composition is
one channel through which principals
influence school quality

40. Principal turnover
• Least effective principals least likely to remain in
a school
– Often transition to other schools, particularly from a
high poverty school
• Little or no evidence that the most
effective principals are disproportionately
likely to leave even high poverty schools