Fifty Years Since The Coleman
Report:
Lessons From Seasonal Comparison
Research
by
Douglas B. Downey
The Traditional Narrative
Disadvantaged children typically
attend poorer schools than
advantaged children and so school
reform is a promising mechanism for
reducing inequality.
The Traditional Narrative:
“One implication stands out
above all: That schools bring little
influence to bear on a child’s
achievement that is independent
of his background and general
social context...” (Coleman et al.
1966:325)
What Role Do Schools Play in
the Stratification System?
ReflectExacerbate
Coleman/
Jencks
Capitalist economy—
Bowles & Gintis
School resources— Hedges
et. al.
Savage inequalities--
Kozol
Cultural capital--
Bourdieu
Early
childhood--
Heckman
Seasonal
Comparison
studies—
Heyns;
Entwisle and
Alexander;
Downey et. al
Compensate
Ability grouping/tracking
—Oakes/Lucas
Limitations of Traditional
School-Based Research
• Struggles to isolate school
effects
• One-sided
• Lacks context
% waking hours 18 year-old
Nicholas spent in school
The Challenge of Isolating Teacher
Effects: A Golf Analogy
A B
Skills after 10 weeks
Time practicing outside of instruction
Beginning skills
Limitations of Traditional
School-Based Research
• Struggles to isolate school
effects
• One-sided
• Lacks context
?
How Might Schools
Be Compensatory?
• Age-based grouping
• Targeting resources toward
disadvantaged children
• Less inequality in vs. out of
school
Limitations of Traditional
School-Based Research
• Struggles to isolate school
effects
• One-sided
• Lacks context
School
Environments
Good schools
Poor schools
Figure 1. A Contextual View: Variation in Non-School and
School Environments
School
Environments
Good schools
Poor schools
Non-school
environments
Excellent
homes
Poor homes
What Role Do Schools Play
in the Stratification System?
• Cognitive skills
• Health
• Behavioral skills
• Network resources
• Earnings
• Crime
Outcomes:
• Socioeconomic
• Race/ethnicity
• Gender
• Sexual orientation
• Parental structure
• Immigrant status
Social groups:
CognitiveSkills Seasonal Comparison Studies: SES Gaps in Cognitive
Skills, ECLS-K (Downey et. al. 2004, 2016)
Kindergarten 1st
GradeSummer
Hi SES
Lo SES
Other Troubling
Empirical Patterns
School Impact
School PeriodNon-school
Period
CognitiveSkills
Impact
Achievement Growth Impact
Achievement Growth Impact
“Are ‘Failing’ Schools Really Failing?”
(Downey, von Hippel, and Hughes 2008)
Achievement Growth Impact
74%
17%
Correlations among measures of school “quality”
and school-level disadvantage
% free lunch
%
disadvantaged
minorities
Achievement
-.38*** -.45***
Growth
.09 -.28***
Impact .01 -.15
What explains the
mediocre performance
of U.S. students on
international tests?
2009 PISA Reading Scores, Age 15
(Merry, 2013)
U.S. Australia Canada
.30 standard deviation units
1998 PPVT Scores, Ages 4-5
U.S. Australia Canada
.31 standard deviation units
Country Infant
Mortality
(rate per
1,000
births)
Low birth
weight %
Poverty
rates
Gini
Inequality
U.S.
2012
5.98 8.0 15.1 .45
Canada
2012
4.85 6.0 9.4 .32
Country Social Welfare
Expenditures
(%GDP)
Public Health
Spending (%
Total Health
Spending)
Unemployment
Insurance for
Family of Four
Replacement
(%)
Unemployment
Insurance
Duration
(weeks)
U.S. 2000 14.2 44.2 57.1 26
Canada
2000
17.3 70.9 76.2 38
Lessons
• We need to know how large achievement
gaps are *before* formal schooling
• We need methods that separate school
from non-school effects (seasonal
comparisons)
• A contextual analysis, focusing on the
counterfactual “What would inequality
look like if there were no schools,” has
value.
School Policy
Recommendations
• End current accountability schemes
• Expand current policies that are
compensatory (e.g., age-based
grouping, equal funding,
standardization, school exposure)
School Research
Recommendations
• Continue to study schools, exploring
how to make them *more*
compensatory
• Rigorously isolate school from non-
school effects
• Link school outcomes to broader
social conditions
Broader Implications
• School reform may be a limited tool
• Improving the educational
performance of disadvantaged
students requires broader social
reform (universal health care, more progressive taxes,
higher minimum wage, stronger unions, Family Leave,
Reform anti-trust, end mass incarceration)
• These broader reforms are
EDUCATION policies
Conclusion:
• The traditional narrative about schools,
that they promote inequality, is
contradicted by seasonal evidence
• Reducing inequality in children’s broader
social conditions is key to reducing
achievement gaps
Should we believe seasonal
comparison results?
• Ceiling effects
• Assumptions
• Replication
High-Flying Schools
Traditional interpretation My interpretation
Disadvantaged children endure
substantially poorer schools
than advantaged children.
Disadvantaged children attend schools
that promote learning at about the
same level as schools serving
advantaged children.
Once disadvantaged children
enjoy similar schools, they start
to close the achievement gaps.
Once disadvantaged children are
exposed to schools substantially better
than those serving advantaged
children, they begin to close
achievement gaps.
We just need to scale up these
small-scale successes.
It would very difficult to scale up a
school system where disadvantaged
children enjoy substantially better
schools than advantaged children.

學校與不均等的關係:美國長期比較研究的經驗談

  • 1.
    Fifty Years SinceThe Coleman Report: Lessons From Seasonal Comparison Research by Douglas B. Downey
  • 2.
    The Traditional Narrative Disadvantagedchildren typically attend poorer schools than advantaged children and so school reform is a promising mechanism for reducing inequality. The Traditional Narrative:
  • 3.
    “One implication standsout above all: That schools bring little influence to bear on a child’s achievement that is independent of his background and general social context...” (Coleman et al. 1966:325)
  • 4.
    What Role DoSchools Play in the Stratification System? ReflectExacerbate Coleman/ Jencks Capitalist economy— Bowles & Gintis School resources— Hedges et. al. Savage inequalities-- Kozol Cultural capital-- Bourdieu Early childhood-- Heckman Seasonal Comparison studies— Heyns; Entwisle and Alexander; Downey et. al Compensate Ability grouping/tracking —Oakes/Lucas
  • 5.
    Limitations of Traditional School-BasedResearch • Struggles to isolate school effects • One-sided • Lacks context
  • 6.
    % waking hours18 year-old Nicholas spent in school
  • 7.
    The Challenge ofIsolating Teacher Effects: A Golf Analogy A B Skills after 10 weeks Time practicing outside of instruction Beginning skills
  • 8.
    Limitations of Traditional School-BasedResearch • Struggles to isolate school effects • One-sided • Lacks context
  • 11.
  • 12.
    How Might Schools BeCompensatory? • Age-based grouping • Targeting resources toward disadvantaged children • Less inequality in vs. out of school
  • 13.
    Limitations of Traditional School-BasedResearch • Struggles to isolate school effects • One-sided • Lacks context
  • 14.
  • 15.
    Figure 1. AContextual View: Variation in Non-School and School Environments School Environments Good schools Poor schools Non-school environments Excellent homes Poor homes
  • 16.
    What Role DoSchools Play in the Stratification System? • Cognitive skills • Health • Behavioral skills • Network resources • Earnings • Crime Outcomes: • Socioeconomic • Race/ethnicity • Gender • Sexual orientation • Parental structure • Immigrant status Social groups:
  • 18.
    CognitiveSkills Seasonal ComparisonStudies: SES Gaps in Cognitive Skills, ECLS-K (Downey et. al. 2004, 2016) Kindergarten 1st GradeSummer Hi SES Lo SES
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
    “Are ‘Failing’ SchoolsReally Failing?” (Downey, von Hippel, and Hughes 2008) Achievement Growth Impact 74% 17%
  • 24.
    Correlations among measuresof school “quality” and school-level disadvantage % free lunch % disadvantaged minorities Achievement -.38*** -.45*** Growth .09 -.28*** Impact .01 -.15
  • 25.
    What explains the mediocreperformance of U.S. students on international tests?
  • 26.
    2009 PISA ReadingScores, Age 15 (Merry, 2013) U.S. Australia Canada .30 standard deviation units
  • 27.
    1998 PPVT Scores,Ages 4-5 U.S. Australia Canada .31 standard deviation units
  • 28.
    Country Infant Mortality (rate per 1,000 births) Lowbirth weight % Poverty rates Gini Inequality U.S. 2012 5.98 8.0 15.1 .45 Canada 2012 4.85 6.0 9.4 .32
  • 29.
    Country Social Welfare Expenditures (%GDP) PublicHealth Spending (% Total Health Spending) Unemployment Insurance for Family of Four Replacement (%) Unemployment Insurance Duration (weeks) U.S. 2000 14.2 44.2 57.1 26 Canada 2000 17.3 70.9 76.2 38
  • 30.
    Lessons • We needto know how large achievement gaps are *before* formal schooling • We need methods that separate school from non-school effects (seasonal comparisons) • A contextual analysis, focusing on the counterfactual “What would inequality look like if there were no schools,” has value.
  • 31.
    School Policy Recommendations • Endcurrent accountability schemes • Expand current policies that are compensatory (e.g., age-based grouping, equal funding, standardization, school exposure)
  • 33.
    School Research Recommendations • Continueto study schools, exploring how to make them *more* compensatory • Rigorously isolate school from non- school effects • Link school outcomes to broader social conditions
  • 35.
    Broader Implications • Schoolreform may be a limited tool • Improving the educational performance of disadvantaged students requires broader social reform (universal health care, more progressive taxes, higher minimum wage, stronger unions, Family Leave, Reform anti-trust, end mass incarceration) • These broader reforms are EDUCATION policies
  • 36.
    Conclusion: • The traditionalnarrative about schools, that they promote inequality, is contradicted by seasonal evidence • Reducing inequality in children’s broader social conditions is key to reducing achievement gaps
  • 38.
    Should we believeseasonal comparison results? • Ceiling effects • Assumptions • Replication
  • 39.
    High-Flying Schools Traditional interpretationMy interpretation Disadvantaged children endure substantially poorer schools than advantaged children. Disadvantaged children attend schools that promote learning at about the same level as schools serving advantaged children. Once disadvantaged children enjoy similar schools, they start to close the achievement gaps. Once disadvantaged children are exposed to schools substantially better than those serving advantaged children, they begin to close achievement gaps. We just need to scale up these small-scale successes. It would very difficult to scale up a school system where disadvantaged children enjoy substantially better schools than advantaged children.

Editor's Notes

  • #2 Thank you for inviting me here. I have very much enjoyed a few days traveling around Taipei and enjoying this beautiful city. My wife and I have been treated to your hot springs, trips around your beautiful campus, a day biking on your trail near the river, a day hiking elephant mountain, a trip to Keelung harbor, Jiufen village, and many good meals. Thank you for your hospitality. So I’m going to speak about schools and inequality and I’m especially excited about this research because I think it has implications for policy. Much of the academic work we do is discussed in academic circles but has little reason to influence policy. But what I’m going to talk about today is potentially very important to policymakers. Perhaps you have heard the story about the man on hands and knees, looking for something….. I’m going to make the case today that this is similar to how we think about schools and inequality. When we observe a large achievement gap between low- and high-income children, we (in the U.S) have typically looked at schools as the source of this problem. But we may be looking in the wrong place, blaming schools when we should be looking elsewhere. Now I won’t let schools off the hook completely, schools are far from perfect, but I am going to make the case today that if we are serious about improving the educational outcomes of disadvantaged children, that we should employ methods that confidently tell us how schools matter. When we do this in the U.S., we end up with a different idea about schools and inequality than what I call the “Traditional Narrative” And it has important implications for the kind of school research that will have the most value.
  • #3 The traditional narrative about schools and inequality is a critical one. Disadvantaged children typically attend poorer schools than advantaged children and so school reform is a promising mechanism for reducing inequality. This is not a controversial position. High-income people know that the schools the poor attend are poor schools, this is part of the reason they buy houses in expensive neighborhoods with good school districts. Poor people know it too, this is part of the reason they search for alternatives to their neighborhood schools. And I see this traditional narrative dominating the Sociological discussions of schools and, ultimately the public conversation and policy. I say this for three reasons: Undergraduate textbooks primarily teach the traditional narrative. They may discuss alternatives but they end up largely presenting a critical view of schools. The manuscripts I review while at ASR, SOE, nearly all either explicitly or implicitly endorse this narrative. The graduate students who arrive at our program universally endorse this perspective. I would go as far as to say that this traditional narrative about schools and inequality has become a cultural assumption among researchers in the U.S. Researchers no longer question it—they simply state it as fact and then go on to study details of the narrative. This narrative is certainly the one I endorsed early in my career. And so part of what you’re going to hear today is my own personal intellectual journey away from this narrative. That journey starts with an article I read in 1992—Summer Setback by Entwisle and Alexander. That was the first time I was exposed to seasonal comparison research. I will describe seasonal research in more detail later, but briefly it involves observing how achievement gaps change when school is in versus the summer, when school is out. I didn’t accept Entwisle and Alexander’s arguments right away, I sort of went through stages of grieving. Certainly anger and denial were part of this journey. I didn’t want to shift away from the traditional narrative of schools, in part, because I was a young teacher when I read their article and I had two great class sessions built around it for my Introductory Sociology class. In the first class, we read Savage Inequalities by Jonathan Kozol and talked about how between school differences in funding meant that some students enjoyed better school facilities, smaller classrooms, a better curriculum, and more highly paid teachers. And the second day we talked about within-school mechanisms like cultural capital, ability grouping, and teacher discrimination that reproduced inequality. The students liked these two class sessions. I liked them too. I was early in my career as a teacher and I was reluctant to change these classes that the students and I both enjoyed so much. But if I’m going to tell the story about the relationship between schools and inequality, I should start with the famous report, now 50 years old.
  • #4 James Coleman was commissioned by the Department of Education to study how disparate resources between schools (especially those attended by black and white students) ended up shaping their academic skills. He conducted a massive study of over 600,000 American students and 4,000 schools Most scholars, including Coleman, expected that he would find evidence that disparities in school resources are the culprit. But instead Coleman famously reported that variations in children’s math and reading skills were only weakly related to school characteristics and more strongly related to family characteristics. Unfortunately, this conclusion was widely misinterpreted as claiming that schools do not matter. Instead, Coleman claimed that schools played little role shaping *achievement gaps*, an important distinction. But the notion that schools did not influence inequality much was unexpected and we have spent the last half century sorting things out. The Coleman Report initiated that discussion, but I believe that in the U.S., the debate over how schools matter has been too narrow.
  • #5 The debate about schools’ role in the stratification system was largely between two groups. On one side, was Coleman, along with Jencks who reanalyzed the data and came to the same conclusion, concluded that schools largely “reflect” existing inequalities in society. They do little to change them. Critics pointed out, correctly, that Coleman and Jencks may have left unmeasured the most critical aspects of schools that generate inequality (e.g., teacher quality). In short, there were good methodological reasons for thinking that Coleman and Jencks might be wrong. And the critics began describing a wide range of school mechanisms that exacerbate inequality. Bourdieu described how some children signal affiliation with elites through their dress, style, and speech, and that teachers reward these students for their status position. Other scholars emphasized how curriculum differentiation, through ability grouping and tracking, led students to be exposed to different quality of instruction. And others pointed out how, especially in the U.S., there exists wide discrepancies in the funding available to schools. Because our system relies so heavily on local property taxes, schools in wealthy neighborhoods enjoy considerably more money than schools in poor neighborhoods. These differences in money result in differences in resources and influence the quality of education via class sizes, quality of textbooks, and teacher salary. But I’m going to make the case that our discussion in the U.S has been too narrow, and that schools may be more positive than either of these groups suggest. The reason I think this discussion has been misplaced is because research that employs better methods comes to a different conclusion—that schools are compensatory—they play a role reducing inequality. The primary reason that the compensatory possibility merits our attention, is because there are important limitations to traditional school-based research. I’ll describe these now.
  • #6 One of the first problems is that children are influenced by both homes and schools. We need to be able to confidently isolate the school effect from other factors outside of the school. The magnitude of this problem struck me when I was reading an article by Herbert Walberg a few years ago. Walberg wrote something outrageous. He wrote that the “average 18-year old American has spent just 13 percent of his/her waking hours in school.” I remember saying out loud—that’s wrong. I have two kids and so I took my son Nicholas’s schedule and started to estimate the numbers for him.
  • #7 I estimated that Nick sleeps about 10 hours a night—the kid sleeps a lot—he goes to school 180 days a year for about 7 hours a day. Once the dust settled I estimated that Nick, when he reached age 18, would have spend 15 percent of his waking hours in school. My estimate was a little higher than Walberg’s (15 vs. 13) because I didn’t let Nick miss a single day of school. But the point is not whether it is 15 or 13 percent—the point is that the vast majority of time awake is spent OUTSIDE OF SCHOOL. This is a surprisingly low number and I encourage you to produce the same estimate for your children in Taiwan. Your estimate may be larger than 15 because you have a longer school year, but it will also probably be lower than what you would expect. This wouldn’t be such a big deal if children all went home to the same quality environments. But we know that is not the case. Some children go home to stable, loving homes, with parents that read to them, provide good health care and nutrition. Unfortunately, other children go home to unstable homes that lack consistent nutrition and health care, let alone intellectual stimulation. The differences in the quality of these home environments in the U.S is staggering. --Sociology of Education has generally not taken this seriously enough, myself included. So when we observe that a school serving poor children with poor funding also has poor test scores, we are not sure if the funding caused the lower math/reading skills or if it is merely correlated with the true causal agent. The problem is that children are not randomly assigned to schools and so it becomes very difficult to know, with confidence, whether differences in outcomes are really due to schools. The standard way to address this is by trying to identify all of the differences between children’s home environments, statistically controlling for them, and then assuming that whatever differences are leftover must be a product of the schools. The main limitation of this approach is that we ALWAYS fail to identify all of the non-school factors that influence children’s learning and so we never successfully isolate school effects with this technique. For example, with our typical models that control for SES, race, gender, family structure, sibship size, and many other indicators of the home environment, we cannot explain more than 15% of the variation in children’s learning. Most of why some children learn faster than others goes unmeasured in our large surveys. As a result, studies that attempt to understand how schools matter by using covariates to equalize children’s non-school environments consistently produce biased estimates of how schools matter because they mix school and non-school influences in unknown ways. The most convincing way to address this problem is to randomly assign children to different schools. This approach can help us identify how a single school characteristic matters. For example, the Tennessee Star Project randomly assigned children to small and large classrooms, helping us develop a more reasonable estimate of how classroom size matters. But notice the subtle shift in the question here, it’s no longer about how schools matter, it’s about how a single school characteristic matters. If we wanted to know the effect of SCHOOLING overall on inequality we would need to randomly assign children to experience school or no school and then observe what happens to inequality in each condition. I know of no study that has done this. So we are left with observational approaches. But as I said, my concern is that trying to isolate school effects by statistically controlling for characteristics of the non-school environment nearly always distorts our understanding of how schools matter because these models do not sufficiently account for the 800 pound gorilla in the room—the non-school environment.
  • #8 Consider an analogy that highlights the problem. Suppose we are interested in determining who is the best golf instructor A or B, and we decide that the best way to determine this is to compare the skills of the instructors’ students. Both instructors teach a group of students for 10 weeks, meeting them every Saturday for a two-hour session. After 10 weeks, we measure the students’ skills and learn that those who had instructor A are better golfers than those who had instructor B. If we were choosing instructors for our kids, we might look at this information and conclude that we’d like our kid to sign up with instructor A. But there’s a problem. The two groups of students each instructor received did not start at the same place. The students instructor A received already had their own golf clubs, knew how to grip the club and had been golfing several times already. In contrast, the students assigned to golf instructor B had virtually no beginning golf skills. So an alternative way to compare the golf instructors would be to compare how much their students IMPROVED between the time they started the program and finished. This would be a better measure, and when we evaluate them this way, it now looks like the two instructors are more similar. But there’s an additional problem, in addition to starting with different skills, the two groups also enjoyed different levels of additional golf instruction during the 10-week program. Those assigned to golf instructor A were more likely to golf with their parents during the week, and some of them even went to other golf instructors for more help. Those with instructor B, however, received very little additional golf instruction. Now when we think about which instructor is better, it is much more complicated. Golf instructor B might actually be the better instructor, even though their students have poorer skills at the end of the 10 weeks, and gained at the same rate as the students with instructor A. When we try and understand how schools matter we run into similar problems. Some schools serve children who arrive at the school doors with advanced skills while other schools serve children who begin far behind. In addition, once school begins, some children go home to enivronments that reinforce school goals, while other children endure homes that distract from school goals. I’m going to show later that we often assume that schools serving disadvantaged children are poor schools, but if we measure them fairly, in a way that comes closer to isolating their effects, we find surprisingly little evidence that disadvantaged children endure poorer learning environments than advantaged children.
  • #9 A second limitation of traditional school-based research is that it tends to be one sided. What I mean by this is that the education researchers tend to be critical of schools, and so they put more effort into looking for and describing the processes within schools that make inequality worse than the processes within schools that might be compensatory. While this is understandable, there is value in evaluating schools critically, it can also lead to a one-sided view of schools that may not be accurate.
  • #10 For example, most of us can generate a wide range of school characteristics that we think might make inequality worse. And these may all be accurate.
  • #11 But most studies do not bother to weigh the magnitude of these exacerbatory processes against the magnitude of compensatory processes. Indeed, they do not even know what the compensatory processes might be, in part because their critical perspective on schools makes them less motivated to focus on these.
  • #12 As a result, we don’t know whether exacerbatory processes outweigh compensatory ones, or even what compensatory processes in schools might be because they are largely undertheorized. Indeed, many of you may be wondering, what compensatory processes am I talking about. How could schools possibly help the disadvantaged MORE than the advantaged?
  • #13 I’m going to briefly discuss three possible ways that schools might be compensatory. First, children’s chronological age is the default basis upon which children are grouped in the U.S. and in most countries. I do not disagree with that practice, but note that the result is a powerful mechanism by which children of widely varying skills are exposed to the same curricular challenges. Social scientists talk a lot about curriculum differentiation (via ability grouping and tracking) but surely this practice overwhelms those and results in a powerful force for curriculum consolidation. Second, while social scientists often emphasize the practices that target resources toward advantaged children, we need to remember that there do exist some forces that target resources toward disadvantaged children. For example, if we ask teachers who they devote most of their time to, 80 percent say “struggling students”. And in the U.S, there exist a wide range of federal programs (e.g., Title 1) that focus resources on disadvantaged children. Finally, schools might be compensatory simply because they are less unequal than the non-school environment. This possibility introduces my final limitation of traditional research, that it lacks context.
  • #14 Most school-based research focuses narrowly on processes WITHIN the institution of schools. To really understand schools’ influence on inequality, however, we need to take a step back
  • #15 Most of our education research has focused on this question—what causes variation between good and poor schools. As a result, we tend to focus on the counterfactual—How would a student do if attending school A versus B? There is value in that question. With careful studies that isolate the school effect, we can learn what kinds of school characteristics/processes influence inequality. We might learn, for example, that smaller classrooms end up reducing inequality because poor children benefit the most from them. But it is limited for telling us much about schools’ broader role in the stratification system.
  • #16 Consider this broader view, contextualizing schools within society. Here, there is inequality in schools, but substantially greater inequality outside of schools. In this world, even if schools were unequal, they are an equalizing force because the inequality children experience in schools is less than what they experience outside of schools. In addition, this framing prompts us toward a different counterfactual than the usual, “How would a student do in school A vs. B?” With this broader framing we start to think about a different counterfactual, what would inequality look like if we didn’t have schools? The advantage of this framing is that it will help us identify schools’ real effect on inequality, which will lead to better policy decisions aimed at reducing inequality.
  • #17 So far I’ve talked about the methodological limitations of traditional research on schools. One might reasonably ask, well, how do we do any research then? I admit, doing research that provides leverage on how schools matter is not easy. And the methods I’m going to discuss now, while I think they have important leverage, also have their own limitations. One of those limitations is that it has been difficult to expand this method beyond limited dimensions of inequality. So most of the work has focused on SES/cognitive skills relationship, although we don’t think that is the only thing that matters. So I’m going to explain the logic behind the seasonal comparison method, but before I do I need to share with you one pattern that shapes much of my thinking about schools and inequality.
  • #18 If you forget everything else I say today I want you to remember this graph. Plots cognitive skills on the Y axis. Age along X, for children Of different groups. Over 90 percent of the reading gap is in place by age 3. When it comes to math, 100 percent of the gap is already present at age 3. These patterns indicate that most of the “action” generating achievement gaps occurs in early childhood. It is a sobering pattern and it suggests that, to really prevent large gaps from forming in the first place, we will need reform that targets early life periods, before school starts. But it doesn’t mean that schools have no role. First, some of the gaps (e.g., the reading gap) continues to grow somewhat during the school years and so it is important to know what role schools play in that. I don’t know if you have this kind of information available in Taiwan, but if I wanted to know how schools affect inequality in Taiwan, one of the first things I would do is try and find data that would reveal what this pattern is here. April 2, 2013 Investments in Education May Be Misdirected By EDUARDO PORTER James Heckman is one of the nation’s top economists studying human development. Thirteen years ago, he shared the Nobel for economics. In February, he stood before the annual meeting of the Nebraska Chamber of Commerce and Industry, showed the assembled business executives a chart, and demolished the United States’ entire approach to education. The chart showed the results of cognitive tests that were first performed in the 1980s on several hundred low-birthweight 3-year-olds, who were then retested at ages 5, 8 and 18. Children of mothers who had graduated from college scored much higher at age 3 than those whose mothers had dropped out of high school, proof of the advantage for young children of living in rich, stimulating environments. More surprising is that the difference in cognitive performance was just as big at age 18 as it had been at age 3. “The gap is there before kids walk into kindergarten,” Mr. Heckman told me. “School neither increases nor reduces it.”
  • #19 So by far, most of what generates achievement gaps occurs prior to schooling, but gaps do still increase a bit and so now I’m going to talk about the seasonal comparison method for understanding whether those gaps increase because of school or the non-school environment. Logic. Non-school factors matter in summer. Non-school AND school matter in school. Differences between these patterns represents the influence of schools. Similar to a cross-over design in medical research. Summer divergence. No divergence during school. The patterns suggest that, if we lived in a world w/o schools and gaps diverged at the summer rate during kindergarten and first grade, then inequality would be greater than what we currently observe. As Karl Alexander puts it: “when it comes to inequality, schools are more part of the solution than the problem” Note how this research design more persuasively isolates school effects (does not rely on trying to measure everything about schools and everything about non-school environments that matter, not one-sided (we can see the *overall* effects of both exacerbatory and compensatory effects by observing how achievement gaps change when school is in versus out), and contextual (we get a peak at what inequality look like if children did not attend school) Some people have looked at these patterns and concluded that schools are roughly neutral, kind of what Coleman said, because learning rates during the school year are roughly similar. But this is a mistake. Think of it like a cross-over design in medical research. The proper way to understand the treatment is to compare the school vs. non-school periods. Clearly schools are compensatory, they are reducing the inequality we would observe in their absence. recall the traditional narrative part that disadvantaged children attend poorer schools. That claim is hard to reconcile with these patterns.
  • #20 The seasonal results have not been explained by scholars who believe that schools make inequality worse. But they are not the only empirical patterns that challenge the traditional narrative.
  • #21 I eventually started to worry about the traditional narrative because of the seasonal results and I started to wonder about our assumption that the schools serving disadvantaged children are poorer than the schools serving advantaged kids. The problem is how would we ever test this given the problem I discussed before with the golf instructors—schools serving disadvantaged children receive children with lower skills AND they try and teach them while those children endure substantially worse home environments. Here is the approach I employed with colleagues. We measured how fast a schools’ children learn when they are not in school (in the summer) and then measured how much FASTER they learned when they were in school. We reasoned that the summer rate of learning is a reasonable estimate of the counterfactual—how children would have learned had they not gone to school—and that the difference in the summer and school year learning rates is a good estimate of schools’ impact on learning. We think this measure more persuasively gauges the quality of what goes on inside a school.
  • #22 And then we applied this method to measuring school quality in our study we asked, “Are failing schools really failing?” We first noted how schools vary in terms of their achievement scores at one point in time. At end of first grade.
  • #23 Which schools look like “failing” schools changes in important ways if we evaluate them via a growth model—gains between end of kindergarten and end of first grade.
  • #24 And which schools are “failing” changes even more if we evaluate schools with our “impact” measure, the difference between summer and first grade learning rates. If we are right, current accountability schemes underestimate the performance of schools serving low-SES children. Growth models move in the right direction, but are still probably biased. Schools are not doing just fine. Some are lousy. We are not giving credit where it is due. Market-based reform—depends on good information about school quality but that these markets suffer from significant inefficiencies. Note that we do NOT conclude that all schools are roughly similar in quality. No, there is substantial meaningful variation in our Impact measure. What is different is that our measure of school quality is not related to social class in the way most think. So we’re not saying schools don’t vary in quality—they do—we’re saying that that variance isn’t distributed in the way we think.
  • #25 Assumption—high SES kids enjoy substantially better school learning environment Differences surely much smaller, maybe nonexistent
  • #26 So what I’ve presented so far makes us think more positively of schools, especially those serving the disadvantaged, But there’s still the fact that U.S. children perform at a disappointing level on international tests. We hear these reports every now and then. U.S is in the middle of the pack. Our Secretary of Education comes out and says “We need to do better.” The school explanations emphasize our weak teacher pool, our poor curriculum, our inefficiency. But one of my students looked at the problem differently.
  • #27 Joe Merry compared the reading skills of 15 year olds in the U.S., Australia, and Canada. He found the typical story that makes us frustrated, American students were far behind. .30 standard deviation units or about a year’s worth of learning.
  • #28 But Joe followed these cohorts backwards to when they were 4-5 years old, prior to formal schooling having a chance to matter, and he found that the gap at that point was completely in place, .31 standard deviation units. This pattern prompts us to rethink the sources of the Canada/U.S. gap, and perhaps a wide range of international gaps. If the gap is already there before schools start, then what the broader social conditions that create it? Why are Canadian kids already ahead of Americans at age 5?
  • #29 Joe’s results make us wonder if the mediocre performance of American students has less to do with differences in school quality and more to do with differences in broader social conditions. For example, Canada has lower infant mortality, fewer low birthweight babies, lower poverty rates and less inequality than we have in America.
  • #30 And it’s likely that these outcomes are related to different policy decisions the two countries have made. Canada spends more on social welfare and has more generous unemployment benefits. Joe’s study prompts us to consider how these broader social conditions shape children’s cognitive skills.
  • #31 I think this research agenda has some lessons for researchers interested in improving the school performance of disadvantaged children. First, it is important to recognize that achievement gaps may be large prior to children beginning formal schooling. In the U.S. they are almost entirely formed. Second, if we want to know how schools influence those gaps, we need to employ methods that confidently separate school from non-school effects. Children spend that vast majority of their waking hours outside of school. If we don’t use methods that carefully account for that fact, we will end up with a distorted view of schools. And third, we need to think about how schools fit into broader society. There is good reason to believe that schools may be compensatory, in part, because they are less unequal than non-school environments. This broader approach to understanding schools helps us locate the real source of inequality.
  • #32 End current accountability schemes. In the U.S. many states evaluate schools and teachers on the basis of their children’s test scores or learning. But these approaches to isolate how schools matter via test scores probably underestimate how well teachers serving disadvantaged children are doing. In the U.S. many states try and use market mechanisms to improve schools. They allow parents “choice” of which school to send their kid to, but parents typically have poor information about which schools are improving children’s learning the most. As a result, the market does not operate in the way economists would hope because of poor information. Expand policies that are compensatory. Schools appear to play an important role reducing inequality. We should leverage that more and emphasize compensatory school policies more. These are probably—age-based grouping, equal funding, standardized curriculum, and greater school exposure. In the U.S., our school calendar is based off an old agricultural economy where children did not go to school in the summer because they helped on the farm. We no longer live in that kind of world and we should increase the number of days children are in school. But let’s remember that school reform is likely to have only a modest effect on achievement gaps because of this graph:
  • #34 What does this mean for researchers who study schools and want to see what schools can do to reduce inequality? There is an exciting new area for education researchers to explore: HOW do schools reduce inequality? What are the mechanisms. We know that important compensatory mechanisms exist AND outweigh the exacerbatory school processes because of seasonal comparison research. But we have spent so little effort thinking about what these compensatory school processes are that we know little about how we might be able to make schools MORE compensatory. For example, if schools are compensatory mostly because of age-based grouping, then perhaps we will want to promote that practice further, limiting retention policies and cases in which we advance children above grade levels. But by studying compensatory mechanisms in schools, scholars could provide a more detailed understanding of how schools matter. We need to take a more balanced approach to studying schools. Most of us who study schools and inequality start with the assumption that schools are a big part of the problem—that they generate inequality. This is not entirely wrong—some practices in schools certainly do make inequality worse. But our methods are overshadowing the compensatory practices. We need to employ methods that will more accurately tell us how schools matter. That means that if we continue to use traditional methods, observing children at one point in time, or on a yearly basis, we will likely struggle to separate school from non-school effects. As a result, will struggle to know the REAL sources of inequality because we will mix schools and non-school factors in unknown ways. This is important because it appears from seasonal comparison research that traditional methods distort our understanding of school’s role. Most importantly, we need to think more about inequality observed in schools (like achievement gaps) is linked to broader social inequality. Since so much of the achievement gaps are formed prior to formal schooling, we need to know more about the social conditions that produce large achievement gaps. For example, Sean Reardon at Stanford has shown that in the U.S. the achievement gap between high- and low-income children has increased over time.
  • #35 Indeed, Since 1970, the achievement gap between children in the 90th vs. 10th income percentiles (or rich vs. poor kids) has increased by 40 percent. Note that this growth in achievement gaps has corresponded to a period in the U.S. where income inequality in general has been increasing.
  • #36 But there are broader implications of this work, especially for those interested in improving the lives of disadvantaged children. School reform may be a limited too because gaps are mostly generated prior to formal schooling and then schools are already helping reduce those gaps. That doesn’t mean that schools serving disadvantaged children are all fine, or that all schools are fine. There are many ways in which schools could do a better job. But when we study school’s role influencing achievement gaps carefully, we learn that schools do not play the pernicious role most of us assumed. If we really want to help disadvantaged children do better in school, probably the most efficient way to do it would be to improve their lives OUTSIDE of school. This is what I would call the “low-hanging fruit” way to improve their school performance. It is likely less expensive to prevent these large gaps from emerging in the first place than to try and remediate them after school has started. Some of the broad reforms that might help in the U.S. include universal health care, more progressive taxes, a higher minimum wage, stronger worker unions, better Family leave policies, reforming anti-trust laws, and ending mass incarceration. Education researchers should start to think of broader social policy as EDUCATION policy. We typically think of school policy as simply changing our schools, but improving young children’s health IS an education policy because it likely has such implications for their early deficit.
  • #37 Finally, in conclusion I would say that the traditional narrative about schools, that they promote inequality, is contradicted by seasonal evidence. And while it is possible that future studies looking at other outcomes may find that schools promote inequality, what we currently know (mostly focusing on socioeconomic gaps in math and reading skills) produces a surprisingly positive view of schools’ role—schools appear to rub the rough edges off inequality. Reducing inequality in children’s broader social conditions is probably going to be key to reducing achievement gaps, because these are mostly formed prior to formal schooling. Shey Shey
  • #39 Of course, seasonal comparison studies aren’t perfect and so we need to think about how this approach to schools might be wrong. Ceiling– routing tests, awkard to apply selectively to seasons Assumptions (Spillover, summer as window into no-school world) (3) Patterns vary in magnitude by generally replicate.
  • #40 Of course, seasonal comparison studies aren’t perfect and so we need to think about how this approach to schools might be wrong. Ceiling– routing tests, awkard to apply selectively to seasons Assumptions (Spillover, summer as window into no-school world) (3) Patterns vary in magnitude by generally replicate.