Does Extended Time Improve Students' Performance? Evidence from Catalonia (Paper)
Does Extended Time Improve Students' Performance?
Evidence from Catalonia
Ana Mara Costa Ramon, Laia Navarro-Sola, Patricia de Cea Sarabia
Master Project - MSc. Economics 2013 - 2014
An educational reform implemented in Catalonia suddenly increased by 20% the total
number of hours of class for all primary students in public schools. This paper evaluates the
eect of this extension of the educational time on student's performance with an identi
strategy which exploits the exogenous group-level variation generated across cohorts,
across regions and across types of schools. Using the PISA database and the econometric
nd that there is no conclusive evidence on
this causal relationship. We propose an alternative methodology, the synthetic control ap-proach,
to solve speci
c concerns about the suitability of the control group, thus providing
a new approach to an old topic.
We would like to thank professor Gabrielle Fack, Caterina Calsamiglia, Joan Llull, Jose Garca Montalvo
and Walter Garca-Fontes for their help and guidance.
Education is one of the main priorities of developed societies, facing new challenges and a
dynamic environment that force to continuously rethink the established system. The countries
are investing huge amounts of resources in this area, but little is known about the eectiveness
of the inputs used in the education production function, leaving the
nal decision of investment
to ideological or political reasons.
In this context, there is an increasing support of extending class time among politicians and
policy-makers as a way of improving education. Although there is plenty of correlational evi-dence,
little is known about the causal eects of extending school time on educational attain-ment.
The following is an investigation of the eect of an increase in the number of hours per
day of class on the performance of the students.
We exploit the exogenous variation generated by a policy change in Catalonia (a region of Spain),
known as the sixth hour policy. This reform introduced one extra hour per day, representing
an increase of 20% of the total number of hours per year. The speci
c characteristics of the
policy implementation provide three dierent sources of variation: variation between cohorts,
generated by the sudden implementation, variation between types of schools, since the policy was
only addressed to public schools leaving private schools timetable unchanged, and in last term,
variation across regions, as the reform only aected public schools in Catalonia. These features
allow us to take the policy implementation as a natural experiment and thus, to investigate
more deeply the eects of extending school time.
Using PISA dataset and a dierences-in-dierences econometric strategy, we do not
nd a strong
evidence of the relationship between extending school time and performance improvement. Some
of our results suggest a negative tendency in tests scores of public schools in Catalonia, but due
to data limitations we cannot conclude that this eect is due to the policy introduction.
This research adheres to the literature that analyses the eects of increasing the hours of class
per day, known as the intensive margin. The main contribution of the paper is related to the
nature of the experiment itself, which provides with two dierent control groups that allow
us to compare the results obtained and increase the robustness of our
ndings. Moreover, we
construct a synthetic control group using a methodology for comparative studies that has not
been widely used in the literature about this topic.
The rest of the paper is structured as follows: in Section 2 we present the main
ndings in the
literature related to extending time at school; in Section 3 we describe the background of our
analysis, including the policy implementation, the database used and the descriptive analysis;
in Section 4 we introduce our identi
cation strategy; in Section 5 we present our main results
as well as other extensions; in Section 6 we conduct a series of robustness checks; in Section 7
we perform an alternative analysis using synthetic control methodology; in Section 8 we discuss
nally in Section 9 we present the conclusions.
2 Literature Review
There is a growing movement among politicians and educational reformers who consider increas-ing
time spent at school as an important and necessary policy in order to improve education.
Among others, Barack Obama claimed that the challenges of a new century demand more time
in the classroom1. But this claim is not new; in fact, extending the length of the school year
was a major policy recommendation of the report in 1983 A Nation at Risk.
The important point comes from trying to de
ne what the optimal amount of time spent
in school is and, as a consequence, to know whether it is true that longer schooling days
or years improve academic performance. As discussed by Cabrales (2013), in economics of
education there is a large debate on whether inputs in the production function of education
like student-teacher ratio, teachers' quality or autonomy of schools have eects on students'
achievement. But there is little evidence on time as an input and on its results in terms of
nition of school time itself. First of all, extending the
time of schooling can be dierent depending on whether they extend the school day, referred
in the literature as intensive margin, or the school year, referred as extensive margin. More-over,
there is no agreement on whether more time in school is better or worse for academic
Proponents of extending the day/year often make use of international comparisons. Silva (2007)
provides data of instructional hours and the ranking in PISA Math exam. The
rst four countries
in the ranking oer higher instructional time than for example the US, placed the 24th. One
of the important arguments in favor of extending the schooling time is that inevitably it will
have an impact on learning via higher time on tasks, higher time for covering the curriculum
and the material or also for repeating it (Farbman, 2012). As a direct evidence, according to
the New Teacher Center Surveys (2013), educators across nine states of the US report lacking
time to collaborate and plan, and thus to meet their professional obligations. Other researchers
pointed out the importance of additional time to improve coordination among professors, as
well as better communication between professors and students (Wenglinsky, 2002).
Moreover, Lavy (2010) stresses the fact that adding instructional time in certain subjects can
also create positive externalities, in the sense that it will lead to increasing diculty of the
material covered, and thus advanced coursework. We cannot forget other bene
ts that are
not related to an improvement of the academic achievement. One of the arguments is that
increasing time at school will lower the exposure of vulnerable groups to the risks of their
problematic neighborhoods, and thus it can also reduce crime (Patall et al, 2010).
On the other hand, skeptics about increasing the number of days or hours of schooling argue
that they do not necessarily increase the instructional time or time engaged in learning, so they
1Fox News. http://www.foxnews.com/politics/2009/09/27/obama-proposes-longer-school-day-shorter-
may have no eect on improving academic performance if it is wasted time. They argue that
improving the quality as well as the eciency of instructional time is at least as important as
the quantity of time spent in school (Aronson et al, 1998).
Also opponents argue against the evidence provided by PISA, remarking that the relationship
between time and academic performance is not a direct channel, and many other variables can
aect this relation. As evidenced by Patall et al(2010), four of the
ve nations that scored
below the US on PISA 2003 also were making use of more instructional time, so the argument
that countries above US have more instructional hours can be reversed.
Another argument is related with the fact that extending the schooling time can have as a
consequence an increase of the fatigue, burnout as well as boredom of student (Edwards, F.,
2012). Moreover, teachers can also burnout from extended hours/day of school time. Silva
(2007) provides evidence that in a poll in 1989 it was found that a high percentage of teachers
opposed to extend school time although an increase in wages was oered according to the
additional time spent. Lastly but not less important, we have to take into account the costs
associated with this type of extending time proposals. There are two main costs, political costs
highlighted by Silva (2007) and monetary costs.
We contribute to this debate by providing new evidence on the eects of extending time at
school using a policy change in Catalonia. The key novelty is intrinsic in the policy design: the
availability of two dierent control groups provide us the possibility of comparing the results
The PISA data set also allow us to contrast whether the results are driven by a particular
time period, as we can perform a full assessment2 analysis, comparing the results in 2012 with
the results in 2003 (as tests focused on Maths both years). We also explore the heterogeneity
eects of this policy across subgroups of students, as in the literature it is repeatedly found that
extended time seems to be more positive for vulnerable groups.
Moreover, we introduce a new methodology in this literature: we try to construct a synthetic
control group that provides us with a closer control group, in order to infer the causal eect
of increasing time.
In Section 3.1,we present the details of the policy implementation, in Section 3.2 we describe
the data source, the PISA database, specifying the main methodological issues involved and in
sections 3.3 and 3.4 we report the main characteristics of our sample.
2PISA focus on three domains: reading literacy, mathematical literacy and scienti
c literacy. While the three
domains form the core of each cycle, two-thirds of the assessment time in each cycle will be devoted to a major
3.a The Policy Implementation
The Pacte Nacional per a l'Educacio (PNE) (National Education Pact) was a collaborative
agreement between Catalonia's Government3 and dierent social and political parties related to
educational changes in Catalonia. Signed on March 2006, this educational pact was a consensus
to provide ample support and solid fundamentals to construct a major education law reform,
nally approved in 2009. The reduction of the existing dierences between schools oering
public educational services was one of the main goals, along with guaranteeing social cohesion
and educational equality of opportunities.
This goal was translated to speci
c policies related to a standardization of the timetable and the
school calendar, jointly with other measures. Our policy of interest arises under this framework.
The sixth hour policy established that all primary schools under public educational services
must oer 30 weekly hours of class. In the year 2005-2006, all public schools, which accounted
for 61.73% of students in Catalonia4, were oering the 25 mandatory curricular hours but the
private government-funded schools were already oering 6 daily hours of class. In the year of
2006-2007, the PNE start the generalization of the 30 hours per week structure to the totality
of Catalan students, supposing a considerable increase of approximately 20% of the total hours
of class for all students in public schools5.
The aim of this policy was to increase the performance in multidisciplinary abilities and cognitive
skills by introducing a complementary hour of class per day, with explicit indications not to
use these extra hours to extend curriculum of any subject. The schools were encouraged to
implement this extra daily hour before the lunch break or added at the end of the day.
c feature of the implementation of the policy is that it was implemented in two dif-ferentiated
waves. In the school year of 2006-2007 the measure was applied to the schools in
the capitals of each sub-region (comarques) and schools in cities with more than 10.000 inhabi-tants.
In the year of 2007-2008, the measure was extended to all remaining schools in Catalonia.
Therefore, the application was neither random nor systematic, with dierences across regions
and across time. Exceptions to postpone or advance one year the policy implementation should
be requested with a previous inspection and reports with objective reasons defending it6. Thus,
although being plausible, we believe that administrative costs make the generalization of im-perfect
compliance fairly unlikely.
3Catalonia is a Spanish region with political competences in the education
eld. There exists three types of
schools: private schools, public schools and private government-funded schools, which although they are managed
privately, they provide public educational service and the government administration provide them with funding
if certain requirements are met.
4Generalitat de Catalunya. Departament d'Educacio i Universitats. Estadstica. Curs 2005-2006
5Under the new school calendar established for the PNE, which de
nes an average of 176 days of school, the
sixth hour policy resulted with students receiving on average 35 days more of class every year and, accumulated
for the six years of primary education, they end up with 1.2 years more on average of primary education.
6Pacte Nacional per a l'Educacio, Generalitat de Catalunya (2006). Annex 1 (p. 55)
The data source used in this study is the PISA database. The OECD Programme for Inter-national
Student Assessment (PISA) is a triennial standardized international survey, started in
2000, whose aim is to evaluate the education worldwide. It is a repeated cross-section database
and the units of observation are students between 15 years and 3 months old and 16 years and
2 months old at the time of the assessment period.
PISA's objective is to evaluate the abilities needed to apply the acquired knowledge to unfamiliar
settings, essential for participation in the society and meeting real-life challenges. The key
cognitive skills evaluated are reading, mathematics and science. More speci
c area of assessment, accounting for approximately a 60% of the total
testing time. The multidisciplinary abilities that PISA seeks to measure are the same skills
aimed to be improved by the sixth hour policy. Hence, even if there has not been much
improvement in other educational areas, the analysis should capture at least the most direct
impact of the policy change.
An advantage of this dataset is that, additional to the test scores, the dataset contains detailed
individual information on student's background, familiar and socioeconomic characteristics and
learning experiences. PISA results in each subject are mapped on a scale curved as a normal
distribution, with an international mean of 500 and a standard deviation of 100 test-score points,
making easier the comparison across regions and years. More importantly, it also contains the
results of a school-level questionnaire on institutional features and school characteristics, thus
providing relevant data and measures to compare schooling institutions.
It is important to explain the methodology used in PISA since we address this particular issues
throughout the analysis. First of all, the sampling design applied in PISA is done in two
rst schools are sampled and, afterwards, students from these participating schools are
sampled. This particular design, as noted in OECD (2009), increases the standard errors of
the population estimates. Thus, standard errors have to be estimated by replication methods.
There are three types of replication methods for two-stage samples, and we use the Balanced
Repeated Replication method (BRR); in particular, the modi
cation proposed by Fay.
Moreover PISA dataset uses imputation methods7, denoted plausible values for reporting stu-dent
performance (OECD (2009)). Five plausible values are allocated to each student on each
performance scale. Due to this particular presentation of the results, the analysis should be
done for each of the
ve plausible values and then the results should be aggregated to obtain the
nal estimates. Finally, the data should be weighted since students and schools in a particular
country could have dierent probability of selection due to dierential participation rates de-pending
on the type of school or student characteristics and also it is possible that some strata
has been over-sampled for a variety of purposes.
Before stating the econometric strategy, it is important to analyze the descriptive statistics of
7See PISA Data Analysis Manual.
our treatment group, as well as our control groups before and after the implementation of the
3.c Descriptive Statistics Section
There are 20,846 students in the sample for PISA 2012 and 21,358 for PISA 2009. For 2012,
there are 675 students from public schools in Catalonia, and 498 for the private ones; in 2009,
607 in public schools and 447 in private schools. In terms of schools in Catalonia participating
in PISA, we have that for 2009 the number of public schools is 23 and 17 privates, and for 2012,
28 public and 19 private schools.
The variables of interest for our study are the results obtained from the dierent tests taken in
the PISA evaluation. The Table A1 (in the appendix) summarizes the descriptive statistics for
the two groups of schools within Catalonia, private schools (which include government-funded
private schools) and public schools, in PISA 2009 and PISA 2012. The key feature is that
private schools do better than public schools in PISA test for 2009 and for 2012. Moreover,
while in 2012 public schools got worst results, private schools improved their performance.
On the one hand, regarding individual characteristics, in private schools there are fewer re-peaters,
for both years, and both private and public had fewer repeaters for 2012, although
the decrease is more pronounced for private. Focusing on the variables that measure the eco-nomic
background of the students (the ESCS8, and the HISEI9 Index), all of them improved
for private schools, and they did not change or barely improved for public schools from 2009
to 2012. Another important characteristic is that the share of immigrants is much higher for
public schools. In terms of school location there are not big dierences. The only signi
dierence is the presence of more private schools in the city.
On the other hand, if we look at the descriptive statistics for the other control group, public
schools in the Basque Country, we
nd that it does better in Maths but Catalonia does better
in all other areas in PISA 2009. For PISA 2012 we observe the same pattern, although the
Basque Country improved its results in Science. Regarding the student body, there are more
repeaters and fewer immigrants in the Basque Country in both years. We observe no major
dierences neither in the HISEI Index nor in the ESCS, although the last improves in 2012 for
the Basque Country. There are also no big dierences in the number of students coming from
unstructured families. In terms of the school location, there are more schools in the Village in
the Basque Country, and essentially no schools in a large city, as opposed to Catalonia.
8ESCS is the PISA index of economic, social and cultural status. It was created on the basis of the following
variables: the International Socio-Economic Index of Occupational Status (ISEI); the highest level of education
of the student's parents, converted into years of schooling; the PISA index of family wealth; the PISA index of
home educational resources; and the PISA index of possessions related to classical culture in the family home.
9HISEI is the highest educational level of parents in years of education according to ISCED.
3.d Dierences in Means Test
To complement the analysis of the descriptive statistics, we run a mean comparison test between
public schools in Catalonia and the two control groups before the treatment. This analysis will
help to identify the relevant dierences on observable variables among treatment and control
groups before the sixth hour policy.
rst mean test compares the treatment group, public schools in Catalonia, with private
schools in Catalonia. The null hypothesis is the following:
H0 = YPRIV