Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Does Extended Time Improve Students' Performance? 
Evidence from Catalonia 
Ana Mara Costa Ramon, Laia Navarro-Sola, Patri...
ca-tion 
strategy which exploits the exogenous group-level variation generated across cohorts, 
across regions and across ...
cation of dierences-in-dierences, we
nd that there is no conclusive evidence on 
this causal relationship. We propose an alternative methodology, the synthetic...
c concerns about the suitability of the control group, thus providing 
a new approach to an old topic. 
We would like to t...
Contents 
1 Introduction 3 
2 Literature Review 4 
3 Background 5 
3.a The Policy Implementation . . . . . . . . . . . . ....
cation Strategy 10 
5 Results 12 
5.a Baseline Regression Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...
1 Introduction 
Education is one of the main priorities of developed societies, facing new challenges and a 
dynamic envir...
nal decision of investment 
to ideological or political reasons. 
In this context, there is an increasing support of exten...
c characteristics of the 
policy implementation provide three dierent sources of variation: variation between cohorts, 
ge...
nd a strong 
evidence of the relationship between extending school time and performance improvement. Some 
of our results ...
ndings. Moreover, we 
construct a synthetic control group using a methodology for comparative studies that has not 
been w...
ndings in the 
literature related to extending time at school; in Section 3 we describe the background of our 
analysis, i...
cation strategy; in Section 5 we present our main results 
as well as other extensions; in Section 6 we conduct a series o...
ndings of our paper, and
nally in Section 9 we present the conclusions. 
3
2 Literature Review 
There is a growing movement among politicians and educational reformers who consider increas-ing 
tim...
ne what the optimal amount of time spent 
in school is and, as a consequence, to know whether it is true that longer schoo...
rst complexity comes from the de
nition of school time itself. First of all, extending the 
time of schooling can be dierent depending on whether they exte...
rst four countries 
in the ranking oer higher instructional time than for example the US, placed the 24th. One 
of the imp...
ts that are 
not related to an improvement of the academic achievement. One of the arguments is that 
increasing time at s...
may have no eect on improving academic performance if it is wasted time. They argue that 
improving the quality as well as...
ve nations that scored 
below the US on PISA 2003 also were making use of more instructional time, so the argument 
that c...
c literacy. While the three 
domains form the core of each cycle, two-thirds of the assessment time in each cycle will be ...
3.a The Policy Implementation 
The Pacte Nacional per a l'Educacio (PNE) (National Education Pact) was a collaborative 
ag...
nally approved in 2009. The reduction of the existing dierences between schools oering 
public educational services was on...
c policies related to a standardization of the timetable and the 
school calendar, jointly with other measures. Our policy...
c feature of the implementation of the policy is that it was implemented in two dif-ferentiated 
waves. In the school year...
eld. There exists three types of 
schools: private schools, public schools and private government-funded schools, which al...
Upcoming SlideShare
Loading in …5
×

Does Extended Time Improve Students' Performance? Evidence from Catalonia (Paper)

596 views

Published on

Published in: Economy & Finance
  • Be the first to comment

  • Be the first to like this

Does Extended Time Improve Students' Performance? Evidence from Catalonia (Paper)

  1. 1. Does Extended Time Improve Students' Performance? Evidence from Catalonia Ana Mara Costa Ramon, Laia Navarro-Sola, Patricia de Cea Sarabia Master Project - MSc. Economics 2013 - 2014 Abstract An educational reform implemented in Catalonia suddenly increased by 20% the total number of hours of class for all primary students in public schools. This paper evaluates the eect of this extension of the educational time on student's performance with an identi
  2. 2. ca-tion strategy which exploits the exogenous group-level variation generated across cohorts, across regions and across types of schools. Using the PISA database and the econometric speci
  3. 3. cation of dierences-in-dierences, we
  4. 4. nd that there is no conclusive evidence on this causal relationship. We propose an alternative methodology, the synthetic control ap-proach, to solve speci
  5. 5. c concerns about the suitability of the control group, thus providing a new approach to an old topic. We would like to thank professor Gabrielle Fack, Caterina Calsamiglia, Joan Llull, Jose Garca Montalvo and Walter Garca-Fontes for their help and guidance. 1
  6. 6. Contents 1 Introduction 3 2 Literature Review 4 3 Background 5 3.a The Policy Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.b Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.c Descriptive Statistics Section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.d Dierences in Means Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4 Identi
  7. 7. cation Strategy 10 5 Results 12 5.a Baseline Regression Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5.b Full Assessment Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 5.c Heterogeneity Eects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 6 Robustness Checks 16 6.a Placebo Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 6.b Parallel Trend Assumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 7 Synthetic Control Method 18 8 Discussion 20 8.a Teacher Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.b Composition Eects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 8.c Implementation of Other Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 8.d Imperfect Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 8.e Short-Term Eects Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 8.f Further Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 9 Conclusion 23 10 Appendix 26 2
  8. 8. 1 Introduction Education is one of the main priorities of developed societies, facing new challenges and a dynamic environment that force to continuously rethink the established system. The countries are investing huge amounts of resources in this area, but little is known about the eectiveness of the inputs used in the education production function, leaving the
  9. 9. nal decision of investment to ideological or political reasons. In this context, there is an increasing support of extending class time among politicians and policy-makers as a way of improving education. Although there is plenty of correlational evi-dence, little is known about the causal eects of extending school time on educational attain-ment. The following is an investigation of the eect of an increase in the number of hours per day of class on the performance of the students. We exploit the exogenous variation generated by a policy change in Catalonia (a region of Spain), known as the sixth hour policy. This reform introduced one extra hour per day, representing an increase of 20% of the total number of hours per year. The speci
  10. 10. c characteristics of the policy implementation provide three dierent sources of variation: variation between cohorts, generated by the sudden implementation, variation between types of schools, since the policy was only addressed to public schools leaving private schools timetable unchanged, and in last term, variation across regions, as the reform only aected public schools in Catalonia. These features allow us to take the policy implementation as a natural experiment and thus, to investigate more deeply the eects of extending school time. Using PISA dataset and a dierences-in-dierences econometric strategy, we do not
  11. 11. nd a strong evidence of the relationship between extending school time and performance improvement. Some of our results suggest a negative tendency in tests scores of public schools in Catalonia, but due to data limitations we cannot conclude that this eect is due to the policy introduction. This research adheres to the literature that analyses the eects of increasing the hours of class per day, known as the intensive margin. The main contribution of the paper is related to the nature of the experiment itself, which provides with two dierent control groups that allow us to compare the results obtained and increase the robustness of our
  12. 12. ndings. Moreover, we construct a synthetic control group using a methodology for comparative studies that has not been widely used in the literature about this topic. The rest of the paper is structured as follows: in Section 2 we present the main
  13. 13. ndings in the literature related to extending time at school; in Section 3 we describe the background of our analysis, including the policy implementation, the database used and the descriptive analysis; in Section 4 we introduce our identi
  14. 14. cation strategy; in Section 5 we present our main results as well as other extensions; in Section 6 we conduct a series of robustness checks; in Section 7 we perform an alternative analysis using synthetic control methodology; in Section 8 we discuss the main
  15. 15. ndings of our paper, and
  16. 16. nally in Section 9 we present the conclusions. 3
  17. 17. 2 Literature Review There is a growing movement among politicians and educational reformers who consider increas-ing time spent at school as an important and necessary policy in order to improve education. Among others, Barack Obama claimed that the challenges of a new century demand more time in the classroom1. But this claim is not new; in fact, extending the length of the school year was a major policy recommendation of the report in 1983 A Nation at Risk. The important point comes from trying to de
  18. 18. ne what the optimal amount of time spent in school is and, as a consequence, to know whether it is true that longer schooling days or years improve academic performance. As discussed by Cabrales (2013), in economics of education there is a large debate on whether inputs in the production function of education like student-teacher ratio, teachers' quality or autonomy of schools have eects on students' achievement. But there is little evidence on time as an input and on its results in terms of academic performance. The
  19. 19. rst complexity comes from the de
  20. 20. nition of school time itself. First of all, extending the time of schooling can be dierent depending on whether they extend the school day, referred in the literature as intensive margin, or the school year, referred as extensive margin. More-over, there is no agreement on whether more time in school is better or worse for academic performance. Proponents of extending the day/year often make use of international comparisons. Silva (2007) provides data of instructional hours and the ranking in PISA Math exam. The
  21. 21. rst four countries in the ranking oer higher instructional time than for example the US, placed the 24th. One of the important arguments in favor of extending the schooling time is that inevitably it will have an impact on learning via higher time on tasks, higher time for covering the curriculum and the material or also for repeating it (Farbman, 2012). As a direct evidence, according to the New Teacher Center Surveys (2013), educators across nine states of the US report lacking time to collaborate and plan, and thus to meet their professional obligations. Other researchers pointed out the importance of additional time to improve coordination among professors, as well as better communication between professors and students (Wenglinsky, 2002). Moreover, Lavy (2010) stresses the fact that adding instructional time in certain subjects can also create positive externalities, in the sense that it will lead to increasing diculty of the material covered, and thus advanced coursework. We cannot forget other bene
  22. 22. ts that are not related to an improvement of the academic achievement. One of the arguments is that increasing time at school will lower the exposure of vulnerable groups to the risks of their problematic neighborhoods, and thus it can also reduce crime (Patall et al, 2010). On the other hand, skeptics about increasing the number of days or hours of schooling argue that they do not necessarily increase the instructional time or time engaged in learning, so they 1Fox News. http://www.foxnews.com/politics/2009/09/27/obama-proposes-longer-school-day-shorter- summer-vacation/ 4
  23. 23. may have no eect on improving academic performance if it is wasted time. They argue that improving the quality as well as the eciency of instructional time is at least as important as the quantity of time spent in school (Aronson et al, 1998). Also opponents argue against the evidence provided by PISA, remarking that the relationship between time and academic performance is not a direct channel, and many other variables can aect this relation. As evidenced by Patall et al(2010), four of the
  24. 24. ve nations that scored below the US on PISA 2003 also were making use of more instructional time, so the argument that countries above US have more instructional hours can be reversed. Another argument is related with the fact that extending the schooling time can have as a consequence an increase of the fatigue, burnout as well as boredom of student (Edwards, F., 2012). Moreover, teachers can also burnout from extended hours/day of school time. Silva (2007) provides evidence that in a poll in 1989 it was found that a high percentage of teachers opposed to extend school time although an increase in wages was oered according to the additional time spent. Lastly but not less important, we have to take into account the costs associated with this type of extending time proposals. There are two main costs, political costs highlighted by Silva (2007) and monetary costs. We contribute to this debate by providing new evidence on the eects of extending time at school using a policy change in Catalonia. The key novelty is intrinsic in the policy design: the availability of two dierent control groups provide us the possibility of comparing the results obtained. The PISA data set also allow us to contrast whether the results are driven by a particular time period, as we can perform a full assessment2 analysis, comparing the results in 2012 with the results in 2003 (as tests focused on Maths both years). We also explore the heterogeneity eects of this policy across subgroups of students, as in the literature it is repeatedly found that extended time seems to be more positive for vulnerable groups. Moreover, we introduce a new methodology in this literature: we try to construct a synthetic control group that provides us with a closer control group, in order to infer the causal eect of increasing time. 3 Background In Section 3.1,we present the details of the policy implementation, in Section 3.2 we describe the data source, the PISA database, specifying the main methodological issues involved and in sections 3.3 and 3.4 we report the main characteristics of our sample. 2PISA focus on three domains: reading literacy, mathematical literacy and scienti
  25. 25. c literacy. While the three domains form the core of each cycle, two-thirds of the assessment time in each cycle will be devoted to a major domain. 5
  26. 26. 3.a The Policy Implementation The Pacte Nacional per a l'Educacio (PNE) (National Education Pact) was a collaborative agreement between Catalonia's Government3 and dierent social and political parties related to educational changes in Catalonia. Signed on March 2006, this educational pact was a consensus to provide ample support and solid fundamentals to construct a major education law reform,
  27. 27. nally approved in 2009. The reduction of the existing dierences between schools oering public educational services was one of the main goals, along with guaranteeing social cohesion and educational equality of opportunities. This goal was translated to speci
  28. 28. c policies related to a standardization of the timetable and the school calendar, jointly with other measures. Our policy of interest arises under this framework. The sixth hour policy established that all primary schools under public educational services must oer 30 weekly hours of class. In the year 2005-2006, all public schools, which accounted for 61.73% of students in Catalonia4, were oering the 25 mandatory curricular hours but the private government-funded schools were already oering 6 daily hours of class. In the year of 2006-2007, the PNE start the generalization of the 30 hours per week structure to the totality of Catalan students, supposing a considerable increase of approximately 20% of the total hours of class for all students in public schools5. The aim of this policy was to increase the performance in multidisciplinary abilities and cognitive skills by introducing a complementary hour of class per day, with explicit indications not to use these extra hours to extend curriculum of any subject. The schools were encouraged to implement this extra daily hour before the lunch break or added at the end of the day. A speci
  29. 29. c feature of the implementation of the policy is that it was implemented in two dif-ferentiated waves. In the school year of 2006-2007 the measure was applied to the schools in the capitals of each sub-region (comarques) and schools in cities with more than 10.000 inhabi-tants. In the year of 2007-2008, the measure was extended to all remaining schools in Catalonia. Therefore, the application was neither random nor systematic, with dierences across regions and across time. Exceptions to postpone or advance one year the policy implementation should be requested with a previous inspection and reports with objective reasons defending it6. Thus, although being plausible, we believe that administrative costs make the generalization of im-perfect compliance fairly unlikely. 3Catalonia is a Spanish region with political competences in the education
  30. 30. eld. There exists three types of schools: private schools, public schools and private government-funded schools, which although they are managed privately, they provide public educational service and the government administration provide them with funding if certain requirements are met. 4Generalitat de Catalunya. Departament d'Educacio i Universitats. Estadstica. Curs 2005-2006 5Under the new school calendar established for the PNE, which de
  31. 31. nes an average of 176 days of school, the sixth hour policy resulted with students receiving on average 35 days more of class every year and, accumulated for the six years of primary education, they end up with 1.2 years more on average of primary education. 6Pacte Nacional per a l'Educacio, Generalitat de Catalunya (2006). Annex 1 (p. 55) 6
  32. 32. 3.b Data The data source used in this study is the PISA database. The OECD Programme for Inter-national Student Assessment (PISA) is a triennial standardized international survey, started in 2000, whose aim is to evaluate the education worldwide. It is a repeated cross-section database and the units of observation are students between 15 years and 3 months old and 16 years and 2 months old at the time of the assessment period. PISA's objective is to evaluate the abilities needed to apply the acquired knowledge to unfamiliar settings, essential for participation in the society and meeting real-life challenges. The key cognitive skills evaluated are reading, mathematics and science. More speci
  33. 33. cally, every year, PISA focus on a speci
  34. 34. c area of assessment, accounting for approximately a 60% of the total testing time. The multidisciplinary abilities that PISA seeks to measure are the same skills aimed to be improved by the sixth hour policy. Hence, even if there has not been much improvement in other educational areas, the analysis should capture at least the most direct impact of the policy change. An advantage of this dataset is that, additional to the test scores, the dataset contains detailed individual information on student's background, familiar and socioeconomic characteristics and learning experiences. PISA results in each subject are mapped on a scale curved as a normal distribution, with an international mean of 500 and a standard deviation of 100 test-score points, making easier the comparison across regions and years. More importantly, it also contains the results of a school-level questionnaire on institutional features and school characteristics, thus providing relevant data and measures to compare schooling institutions. It is important to explain the methodology used in PISA since we address this particular issues throughout the analysis. First of all, the sampling design applied in PISA is done in two stages:
  35. 35. rst schools are sampled and, afterwards, students from these participating schools are sampled. This particular design, as noted in OECD (2009), increases the standard errors of the population estimates. Thus, standard errors have to be estimated by replication methods. There are three types of replication methods for two-stage samples, and we use the Balanced Repeated Replication method (BRR); in particular, the modi
  36. 36. cation proposed by Fay. Moreover PISA dataset uses imputation methods7, denoted plausible values for reporting stu-dent performance (OECD (2009)). Five plausible values are allocated to each student on each performance scale. Due to this particular presentation of the results, the analysis should be done for each of the
  37. 37. ve plausible values and then the results should be aggregated to obtain the
  38. 38. nal estimates. Finally, the data should be weighted since students and schools in a particular country could have dierent probability of selection due to dierential participation rates de-pending on the type of school or student characteristics and also it is possible that some strata has been over-sampled for a variety of purposes. Before stating the econometric strategy, it is important to analyze the descriptive statistics of 7See PISA Data Analysis Manual. 7
  39. 39. our treatment group, as well as our control groups before and after the implementation of the policy. 3.c Descriptive Statistics Section There are 20,846 students in the sample for PISA 2012 and 21,358 for PISA 2009. For 2012, there are 675 students from public schools in Catalonia, and 498 for the private ones; in 2009, 607 in public schools and 447 in private schools. In terms of schools in Catalonia participating in PISA, we have that for 2009 the number of public schools is 23 and 17 privates, and for 2012, 28 public and 19 private schools. The variables of interest for our study are the results obtained from the dierent tests taken in the PISA evaluation. The Table A1 (in the appendix) summarizes the descriptive statistics for the two groups of schools within Catalonia, private schools (which include government-funded private schools) and public schools, in PISA 2009 and PISA 2012. The key feature is that private schools do better than public schools in PISA test for 2009 and for 2012. Moreover, while in 2012 public schools got worst results, private schools improved their performance. On the one hand, regarding individual characteristics, in private schools there are fewer re-peaters, for both years, and both private and public had fewer repeaters for 2012, although the decrease is more pronounced for private. Focusing on the variables that measure the eco-nomic background of the students (the ESCS8, and the HISEI9 Index), all of them improved for private schools, and they did not change or barely improved for public schools from 2009 to 2012. Another important characteristic is that the share of immigrants is much higher for public schools. In terms of school location there are not big dierences. The only signi
  40. 40. cant dierence is the presence of more private schools in the city. On the other hand, if we look at the descriptive statistics for the other control group, public schools in the Basque Country, we
  41. 41. nd that it does better in Maths but Catalonia does better in all other areas in PISA 2009. For PISA 2012 we observe the same pattern, although the Basque Country improved its results in Science. Regarding the student body, there are more repeaters and fewer immigrants in the Basque Country in both years. We observe no major dierences neither in the HISEI Index nor in the ESCS, although the last improves in 2012 for the Basque Country. There are also no big dierences in the number of students coming from unstructured families. In terms of the school location, there are more schools in the Village in the Basque Country, and essentially no schools in a large city, as opposed to Catalonia. 8ESCS is the PISA index of economic, social and cultural status. It was created on the basis of the following variables: the International Socio-Economic Index of Occupational Status (ISEI); the highest level of education of the student's parents, converted into years of schooling; the PISA index of family wealth; the PISA index of home educational resources; and the PISA index of possessions related to classical culture in the family home. 9HISEI is the highest educational level of parents in years of education according to ISCED. 8
  42. 42. 3.d Dierences in Means Test To complement the analysis of the descriptive statistics, we run a mean comparison test between public schools in Catalonia and the two control groups before the treatment. This analysis will help to identify the relevant dierences on observable variables among treatment and control groups before the sixth hour policy. The
  43. 43. rst mean test compares the treatment group, public schools in Catalonia, with private schools in Catalonia. The null hypothesis is the following: H0 = YPRIV

×