Do extra curricular activities in schools improve

  • 481 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
481
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
16
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Do extra-curricular activities in schools improve educational outcomes? A critical review and meta-analysis of the literature Boaz Shulruf Published online: 2 February 2011 Ó Springer Science+Business Media B.V. 2011 Abstract Secondary schools tend to sponsor a large number of extra-curricular activities (ECA) yet little is known about their contribution to students’ educational outcomes. This meta-analysis aims to determine what it is about ECA participation that supports positive educational outcomes. Furthermore, this study challenges the theoretical assumptions about the benefits of participation in ECA. 29 studies (all except for one based on data collected in the United States) met the search criteria for inclusion in the analysis. Most effect sizes on academic achievements yielded from non-specific ECA, academic clubs and journalism were small, as were par- ticipation in performing arts, sports and leadership activities on a range of educa- tional outcomes. Although the results show associations between participation in ECA and educational outcomes, causal effects could not be confirmed. It is con- cluded that the lack of evidence supporting the causal effects, and thus the common theoretical assumptions about the effects of ECA on educational outcomes, is due to methodology limitations in these studies. Keywords Extra-curricular activities Á Systematic review Á Secondary schools Á Academic achievements Re´sume´ Les activite´s pe´riscolaires des e´coles ame´liorent-elles les re´sultats des e´le`ves ? Analyse critique et me´ta-analyse de la documentation – Les e´tablissements secondaires ont tendance a` organiser un grand nombre d’activite´s pe´riscolaires, mais la contribution de ces dernie`res aux re´sultats scolaires des e´le`ves est faiblement connue. Cette me´ta-analyse avait pour but de cerner dans quelle mesure la partic- ipation a` ces activite´s qui favorise des re´sultats positifs. D’autre part, cette e´tude remet en question les hypothe`ses the´oriques e´mises sur les be´ne´fices de cette B. Shulruf (&) Centre of Medical and Health Sciences Education, Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand e-mail: b.shulruf@auckland.ac.nz 123 Int Rev Educ (2010) 56:591–612 DOI 10.1007/s11159-010-9180-x
  • 2. participation. Vingt-neuf e´tudes (sauf une fonde´e sur des donne´es collecte´es aux E´tats-Unis) re´pondaient au crite`re de recherche pour eˆtre incluses dans l’analyse. La plupart des tailles d’effets sur les re´sultats scolaires retire´s des activite´s pe´riscolaires non spe´cifiques, par les clubs et le journalisme scolaires sont faibles, il en est de meˆme pour les arts du spectacle, les activite´s sportives et celles favorisant l’esprit d’initiative. Bien que les conclusions de l’analyse e´tablissent des liens entre par- ticipation aux activite´s pe´riscolaires et re´sultats scolaires, un lien de cause a` effet n’a pu eˆtre de´montre´. Nous en de´duisons que le manque de donne´es probantes e´tayant ce lien de cause a` effet, et donc les hypothe`ses the´oriques courantes sur l’impact de ces activite´s sur les re´sultats scolaires, est duˆ aux limites me´thod- ologiques de ces e´tudes. Zusammenfassung Werden schulische Leistungen durch außercurriculare Akti- vita¨ten verbessert? Eine kritische U¨ berpru¨fung und Metaanalyse der Literatur – Les activite´s pe´riscolaires des e´coles ame´liorent-elles les re´sultats des e´le`ves ? Analyse critique et me´ta-analyse de la documentation - Viele Sekundarschulen fo¨rdern eine große Anzahl außercurricularer Aktivita¨ten, u¨ber deren Nutzen fu¨r die schulischen Leistungen der Schu¨lerinnen und Schu¨ler allerdings wenig bekannt ist. Diese Metaanalyse soll daru¨ber Aufschluss geben, inwiefern sich die Teilnahme an au- ßercurricularen Aktivita¨ten gu¨nstig auf die schulischen Leistungen auswirkt. Zudem stellt diese Studie die theoretischen Annahmen u¨ber die Vorteile der Beteiligung an außercurricularen Aktivita¨ten auf den Pru¨fstand. Die Kriterien fu¨r die Aufnahme in die Analyse wurden von 29 Studien (alle außer einer auf der Grundlage von Da- tenerhebungen in den Vereinigten Staaten) erfu¨llt. Unspezifische außercurriculare Aktivita¨ten, Wissenschafts-AGs und journalistische Ta¨tigkeiten hatten meist nur geringe Effekte auf den Schulerfolg. Auch die Beteiligung an Theater-AGs oder sportlichen Aktivita¨ten sowie die U¨ bernahme von Fu¨hrungsaufgaben wirkten sich wenig auf bestimmte schulische Leistungen aus. Aus den Ergebnissen lassen sich zwar gewisse Korrelationen zwischen der Teilnahme an außercurricularen Akti- vita¨ten und schulischen Leistungen ablesen, Kausalzusammenha¨nge konnten jedoch nicht nachgewiesen werden. Die Analyse kommt zu dem Schluss, dass die mangelnde Nachweisbarkeit kausaler Zusammenha¨nge und somit der gemeinsamen theoretischen Annahmen u¨ber die Effekte außercurricularer Aktivita¨ten auf die schulischen Leistungen der eingeschra¨nkten Methodik dieser Studien geschuldet ist. Resumen >Mejoran las actividades extracurriculares en las escuelas los resultados educativos? Revisio´n crı´tica y meta-ana´lisis de la literatura – Las escuelas secun- darias tienden a promover un gran nu´mero de actividades extracurriculares (ECA, por sus siglas en ingle´s). Sin embargo, es poco lo que se sabe sobre su aporte a los resultados educativos de los estudiantes. Este meta-ana´lisis apunta a determinar que´ es lo que sostiene los resultados educativos positivos de una participacio´n en ECA. Adema´s, con este estudio se ponen en tela de juicio las suposiciones teo´ricas en cuanto a los beneficios de una participacio´n en ECA. 29 estudios (todos ellos, con excepcio´n de uno, basados en datos recogidos en los Estados Unidos) reunieron los criterios de bu´squeda para ser incluidos en el ana´lisis. La mayorı´a de los taman˜os del efecto relacionados con logros acade´micos obtenidos a partir de ECA no 592 B. Shulruf 123
  • 3. especı´ficos, clubes acade´micos y periodismo eran pequen˜os, igual que la partici- pacio´n en artes drama´ticas, deportes o actividades de liderazgo sobre una serie de resultados educativos. Si bien los resultados muestran relaciones entre una partic- ipacio´n en ECA y resultados educativos, no se han podido confirmar efectos cau- sales. Se concluye que la falta de pruebas que sostienen los efectos causales, y con ello las suposiciones teo´ricas generales sobre los efectos de ECA sobre resultados educativos, se debe a las limitaciones metodolo´gicas de estos estudios. Student performance at secondary school is the main determinant for enrolment in tertiary education. As part of their school day, many secondary school students take part in a wide variety of activities, over and above the core curricular teaching and learning classroom work, which are usually referred to as co-curricular or extra- curricular activities (ECA). In addressing this demand, whether from students or other stakeholders, secondary schools allocate considerable resources to the extra curriculum. Nonetheless, despite a plethora of research studies on the subject, no conclusive evidence can be drawn from the literature indicating the actual benefit students gain by participating in ECA. This critical meta-analysis of the literature, therefore, looks at the benefits and outcomes for students of their participation in ECA, sponsored by secondary-level schools. It is noted, however, that most of the literature in this area has been generated from data collected in the United States, making this paper particularly relevant to the US context. The aims of this paper are to examine the literature on ECA participation and the outcomes of such participation, and to attempt to identify the extent to which Do extra-curricular activities in schools improve educational outcomes? 593 123
  • 4. participation in ECA affects educational outcomes. This paper not only investigates the effect sizes of participation in ECA on a range of educational outcomes but also looks at the validity of the measures used in previous studies, with particular reference to evidence for causal effects. Theoretical framework Extra-curricular activities are an integral component of school life. While participation rates are known to be very high in the United States (in 1995 approximately 60% of high school sophomores and 70% of seniors participated in at least one activity; Cooper et al. 1999), there is very limited documentation on the extent of participation elsewhere. Investigations into the impact of ECA on students date back as early as the 1930s with studies documenting the range of activities being offered in schools and questioning whether participation in certain high school activities could be related to higher achievement at college (Baxter 1936; Holland 1933). However, despite a raft of studies on ECA participation being undertaken over nearly eight decades (mainly in the United States), little is known or understood about the causal effect between participation in such activities and educational outcomes. In 1987, Holland and Andre carried out a review of literature relating to extra- curricular participation and adolescent development with the aim of providing a critique of methodological approaches and possible directions for future research (Holland and Andre 1987). However, while this review was relatively comprehen- sive, it was said to lack the evidence needed to support the conclusions drawn and, according to Taylor and Chiogioji (1988), it had no theoretical framework or definitions of its key terms. Notwithstanding, this study did raise the profile of extra- curricular activity and exposed it as an area which needed more research. A number of studies, particularly in the last decade, has examined ECA within a school-sponsored context (e.g. Broh 2002; Davalos et al. 1999; McNeal 1995; Silliker and Quirk 1997), and many others have taken broader views in terms of ‘‘extra-curricular’’ definition and have included non-school-sponsored activity, such as community and church-based activities (Eccles and Barber 1999; Gerber 1996; Lewis 2004); organised and non-organised activities for high school students (Chambers and Schreiber 2004); as well as social behaviours and high-risk activities, such as smoking, alcohol consumption and drug taking (Eccles and Barber 1999; Eccles et al. 2003). For the purposes of this study, ECA have been defined as those which are school-sponsored and external to the core curriculum. It is acknowledged that this definition may not be the same as that used by other authors. A recent and comprehensive review of literature on ECA for high school students in the USA (Feldman and Matjasko 2005) found that while ECA are viewed as being highly important ‘‘developmental settings for adolescents’’, little is under- stood about the ‘‘contextual influences’’ affecting that development, or the relationship between participation and outcomes (pp. 160–161). Studies such as Lewis’ (2004) meta-analysis of extra-curricular participation conclude that the best 594 B. Shulruf 123
  • 5. academic and social outcomes for students are gained through well-designed, developmentally appropriate activities but are unable to describe or pinpoint the particular characteristics contributing to these outcomes. Furthermore, there is very limited evidence to support the commonly held justification for carrying out ECA. Extra-curricular studies and previous methodological approaches Extra-curricular studies examine a multitude of activities. Some authors take a broad-based approach and examine participation across a range of activities, both school-sponsored and outside-of-school-time activities; while others nominate and examine particular extra-curricular areas, such as participation in athletics or in academically related activities. Studies which only focus on school-sponsored activities tend to be the exception rather than the rule. However, in the past decade there has been a move towards examining ECA within school programmes and their impact on student learning outcomes. McNeal (1995), for example, examined student involvement in ‘formal’ extra-curricular school activities and their impact on student retention, and Broh (2002) studied the educational impact of participation in both sport-related and non- sport-related school-sponsored activities. Many studies have set out to examine how participation in such activities is beneficial for students (for example Guest and Schneider 2003). Melnick et al. (1992) examined the educational effects of interscholastic participation on African- American and Hispanic boys and girls. They found that while high school athletic participation was for many a means of being included in social youth groups, it was not necessarily related to grades and standardised test scores. Another found a strong link between participation in athletics and the likelihood of students dropping out of school (Mahoney and Cairns 1997; McNeal 1995); and still more have found that participation in sports is linked to improved school attendance, academic outcomes, social relationships and self-esteem (Marsh and Kleitman 2003; McCarthy 2000; Silliker and Quirk 1997). The role which participation in ECA plays in educational attainment and cultural capital gain is also highlighted by the literature (Buoye 2004; Power 1999; Thomas and Moran 1991) as well as the role which cultural capital plays in attaining higher academic ranking (Appelman et al. 2003). Group membership through participation in activities is seen as providing access to the relationships and networks that influence and support positive outcomes for students, as well as improving the opportunities to access knowledge and skills which support higher social and/or cultural status. Other studies have examined areas such as the association between ECA and improved chances for college attendance (Kaufman and Gabler 2004; Mahoney et al. 2003); as well as mentoring and its impact on academic achievement and dropout rates (Slicker and Palmer 1993). In the latter group, effective mentoring was found to significantly support improved academic outcomes and retention, while ineffective mentoring had no effect. Many studies emphasise the impact of moderating factors such as gender, ethnicity and socioeconomic status on the level and type of outcome achieved Do extra-curricular activities in schools improve educational outcomes? 595 123
  • 6. (Braddock II et al. 1991; Brown and Evans 2002; Diaz 2005; Gerber 1996; Jordan 1999; Lisella and Serwatka 1996). Davalos et al. (1999), for example, found that both ethnicity and gender were important moderating factors on the impact of ECA on school retention. On the other hand, in examining the association between ECA and academic achievement, Valentine et al. (2002) stressed the importance of a different range of moderating factors to account for why, how and for whom achievement is supported. These factors included the characteristics of the materials used in activities, group characteristics or distinctiveness, school size and cultural issues. This overview indicates that the research literature on the effects of participation in ECA is broad. Aiming to identify the extent to which participation in ECA affects educational outcomes such as attainment, grades, self-concept, aspirations for higher education and enrolment in higher education institutions, this study focuses mainly on the validity of the measures used in previous studies and on the strength of the evidence suggesting causal effects of participation in ECA on these educational outcomes. Method The systematic approach used to carry out this literature review on extra-curricular activity in the secondary school context is different from the approach taken in traditional descriptive or narrative reviews. While the purposes of a narrative review and a systematic review are ostensibly similar – namely to examine the material pertaining to a particular area – a narrative review does not utilise methods which allow for the control of potential methodological biases occurring in the literature (Boaz et al. 2002). Systematic reviews of literature are carried out to agreed research standards with the aim of supporting as objective an approach as possible and therefore minimising bias. The agreed research standards include: using a protocol to guide and plan the processes to be followed; focusing on a specific question; identifying as much of the relevant literature as possible through a comprehensive search; making decisions about the inclusion and exclusion of studies based on methodological criteria; synthesising research findings and being explicit and transparent (Barr et al. 1999; Boaz et al. 2002). To help reduce bias further, a collaborative team approach was used for all decision-making (Kitchenham 2004). The collaborative team included the author and another two scholars who are familiar with the literature on systematic reviews and educational intervention in compulsory education systems. Although systematic reviews, realist reviews, meta-analyses and best evidence syntheses follow ‘‘highly specified and intentionally inflexible methodology with the aim of assuring high reliability’’ (Pawson et al. 2004, p. 5), the methods employed by each approach have attracted significant critique (Davies 2000; Feinstein 1995; Shapiro 1997; Suri and Clarke 1999; Torgerson 2006). In the extreme, Feinstein (1995) described these approaches (predominantly meta- analyses) as ‘‘statistical alchemy of the twenty-first century’’. Although acknowl- edging the discontent surrounding such strict methodologies, this review adheres to 596 B. Shulruf 123
  • 7. the standards appropriate to meta-analyses and systematic reviews in terms of its search and selection strategies and analysis methods in order to minimise possible biases relating to study selection (Lipsey and Wilson 2001). At the same time, however, it specifically looks for evidence of causality relating to the effect sizes identified in the reviewed studies. This is an important contribution to the literature, particularly since causality is a relatively under-researched topic in meta-analyses of educational studies. In summary, this study systematically reviews quantitative studies examining the causal effects of participation in extra-curricular activity at secondary school level on students’ educational outcomes in the areas of academic achievement, school engagement and retention, future aspirations and student self-concept. Search strategy This study followed the recommendations of the ESRC UK Centre for Evidence- Based Policy and Practice for conducting a systematic review search (Boaz et al. 2002). These recommendations include: performing a literature search which includes both peer-reviewed and non-peer-reviewed literature and encompasses a wide range of sources including large-scale and small-scale electronic databases, grey literature such as practitioner journals, books, government reports; taking a team approach to define keywords and develop exclusion and inclusion criteria and compiling a list of search tools and other means used to search data bases and other search sources. A wide range of academic databases was searched: PsycInfo, ERIC, JSTOR, Proquest, Proquest Social Science Journals, Expanded Academic, Web of Science, Sociological Abstracts and Sports Discus. A search of Google Scholar was also attempted; however, the high number of hits and difficulties in reducing these, even with keyword definition, led to this source (Google Scholar) being excluded. In addition, what is known as a ‘‘grey search’’ was carried out which included evaluation reports and reviews conducted by research institutes (such as the Harvard Family Research Project), as well as studies previously sourced by team members. A search of both Campbell Collaboration and Cochrane Collaboration databases was also conducted and revealed that no reviews have been carried out on the area of ECA. Keyword definition In order to facilitate a comprehensive search of the literature, the review team drew up a list of possible keywords which described the activity, the research terms used in examining effectiveness of outcomes, some outcome terms, and some preliminary exclusion criteria. Thirty-eight keywords were used to describe the activity and included: ECA, out-of-school activities, non-academic school activities, learning support and leisure/recreation activities. Research terms included: evalu- ation, impact, effect, relationship and causal. Outcomes were defined as academic, engagement and retention. The full list of keywords is shown in Table 1. Do extra-curricular activities in schools improve educational outcomes? 597 123
  • 8. Some studies were excluded during the search for the following reasons: (i) if they reported on data stated to have been collected before 1985; (ii) if the involved students were less than 13 years of age or over 19 years of age and (iii) if the reports were not written in English. The resulting comprehensive search identified 136 studies meeting the preliminary criteria, although some flexibility was accepted regarding the age- related exclusion criteria (for example, studies were accepted if they were longitudinal and tracked students from junior high school through to high school). Further inclusion and exclusion criteria were set up to assess the quality of studies and the soundness of their methodology (see Table 2). Studies were then screened using these additional criteria. Search results Out of the 88 studies which met the criteria for population, type of activity and educational outcomes, 58 included quantitative data and were therefore selected for further analysis. A final examination of these studies was carried out during the quantitative data entry process, revealing that only 29 studies described their methodology and findings in sufficient depth to be accepted for further analysis for this review (see Table 3). Each included study is highlighted (*) in the final reference list of this review. Only one study was undertaken in the United Kingdom; the rest originated in the United States. Hence, this study is particularly relevant to the US context. Twenty-three of the studies based their findings on data from secondary sources, which included national longitudinal studies such as the National Education Longitudinal Study in 1988 (NELS:88) (NCES 2006) and High School and Beyond (HSB) (Adelman 1995; Konstantopoulos 2005); and regional longitudinal studies Table 1 Keyword definitions Keyword activity definition Extracurricular activities/engagement; Out-of-school activities; Outside-of-school/outside school; Out-of-school learning activities; Extra class activities School club membership; School sports Non-academic student activities; After-school experiences; After-school clubs; School-sponsored activities; Non-school-sponsored activities; After school; physical activities; Adolescent activities; Leisure/recreation activities; Adult sponsored structured activities; Extramural activities; Activity involvement; Art & physical activities School-based extra-curricular; Non-curricular studies/activities; Youth activities; Homework club; Student engagement; Careers-related activities; Academic enrichment; Extended learning; Non-class experience; Extra classroom factors; Leisure-time physical activity; Out-of-school time use; Leisure time; Structured out-of-school time; SEA/structured extracurricular activities; Learning support; Academic support; Creative arts; Mentoring/student support Research terms Evaluation; Impact; Effect; Outcome; Relationship; Significance; Causal; Indicator; Longitudinal; Benefits; Comparison; Study; Analysis Outcome terms Academic/learning outcome; Future aspirations; Engagement/retention/school holding power; Retention; Self-worth/Self-concept/self-esteem; (incl. development of leadership & responsibility) 598 B. Shulruf 123
  • 9. such as the Carolina Longitudinal Study (Robinson and Etheridge 1995) and the Michigan Study of Adolescent Life Transitions (MSALT) (Barber et al. 2001). All 29 studies focused, to some extent, on school-sponsored ECA. 19 looked only at school-sponsored activities; the remaining studies involved both school- sponsored and non-school-sponsored (but similar) activities. In only four instances did studies look at the impact of total ECA participation in both school-sponsored and non-school sponsored activities; the other 25 studies examined the effect of a range of ECA on a variety of educational outcomes. The ECA included academic, sporting, leadership, performing and creative arts, vocational and other community activities. The outcomes considered included academic achievement and aspiration towards future education or training. School retention or holding power was a further key outcome measured. Studies were only included in the meta-analysis if the methodology and data used enabled fair assessment of their results. Furthermore, only studies providing statistical data which could be modified to effect sizes were included. It is acknowledged that adherence to these strict inclusion criteria limits the breadth of the review, and that it is possible that the omission of studies not meeting any of the criteria might bias the meta-analysis (Feinstein 1995; Suri and Clarke 1999). Nonetheless, since the focus of the current study is on the causal effects of ECA, it was essential to establish, and adhere to, these restrictive selection criteria. It is Table 2 Inclusion/exclusion criteria Criteria Description Study design Quantitative methods included controlled observational studies, studies with qualitative description supported by other systematic data collection and studies with qualitative perceptions only (teacher reports only were excluded) Research measures Data-collection tools were standardised, supporting reliability and validity of data Sampling Representative sampling and adequacy of sample size were included as criteria Rigor of measurement Studies needed to be sufficiently transparent to allow judgments about the merits of assessment and the adequacy of data on moderating factors such as ethnicity, gender, type of activity, peer pressure, number of activities, fit between participants and activity, or external factors. Transparency also enabled examination of whether findings were substantiated by data Data collection and analysis Clear, explicit description of data collection and analysis including data on sampling methods and response rate Type of activity Structured, school-sponsored/school-based; extra to core curriculum activities; not employment, work-related Type of outcome Academic/learning outcome; future aspirations; engagement/retention/school holding power; self worth/self concept/self esteem (including development of leadership/responsibility) Academic/learning outcome Future aspirations; engagement/retention/school holding power; self worth/self concept/self esteem (including development of leadership/responsibility) Context Adequate description was essential with aims and objectives clearly described Theoretical underpinning Clear and transparent theory was important but not essential Note: Studies were included if they met the majority of the following criteria for both quality of meth- odology and focus of the study Do extra-curricular activities in schools improve educational outcomes? 599 123
  • 10. suggested, however, that other meta-analyses looking at associations rather than causal effects could take a much broader approach to their selection criteria. Limitations As previously noted, the majority of literature which discusses and investigates ECA has been undertaken in the USA where such activities are largely externally contracted rather than run by teachers working in the schools themselves. Many of the studies are based on secondary data sources involving large national data sets from longitudinal studies, such as the data from High School and Beyond Study, and NELS (e.g. Broh 2002; Darling et al. 2005; Marsh 1992; McNeal 1995; Melnick et al. 1992). The use of secondary data raises the issue of applicability, since the original data sets were not developed with the purpose of testing the impact of participation in ECA. As a result, these studies, while valuable, tend to adopt a similar methodology, highlighting associations between participation and outcomes with analysis around the pre-defined moderating variables measured in the original study, such as peer relationships, race and gender. As these studies rarely have the opportunity to deal with the original data, this presents a potentially confounding factor in the accuracy of evidence. In addition, the large number of studies using the same dataset (although utilising somewhat different measures and sub-samples), has the potential to bias the outcomes of any pooled effect sizes. A further limitation noted is associated with the common use of secondary data sources where measures are defined for national or large-scale data collection purposes. Many of the reviewed studies define educational outcomes in terms of Table 3 Studies selection Database Initial hits Included in the first stage Included in the second stage (met the criteria for population, type of activity and outcomes) Selected for quantitative analysis (included quantitative data allowing calculation of effect sizes) PsychInfo 372 9 >>>>>>>>>>>>>>>>>>>>>>>>>= >>>>>>>>>>>>>>>>>>>>>>>>>; 136 88 29 ERIC 141 JSTOR 7 Snowballing (grey literature) 58 Expanded academic 390 Proquest Social Science Journals 23 Proquest 13 Web of Science 24 Sociological abstracts advanced search 19 Sport Discus 32 600 B. Shulruf 123
  • 11. academic achievement focusing on Grade Point Averages (GPAs) and scores (Branch 2003; Chambers and Schreiber 2004; Keith et al. 2004; Lipscomb 2007; Marsh 1992; Silliker and Quirk 1997); however, Ainley (1994) commented that one of the issues in studies examining school effectiveness is the emphasis on limited ranges of achievement measures (i.e. achievement test scores in core curriculum areas) as indicators of school quality. Ainley (of the Australian Council for Educational Research), suggested that attention to other indicators, such as student engagement, might have been more appropriate and might provide a better understanding of what shapes student outcomes. Although worthwhile in terms of contributing to the pool of information on the effect of ECA on student outcomes, few of the studies were positioned within a coherent theoretical framework. In their recent review of the literature on ECA in high school, Feldman and Matjasko (2005) commented that ‘‘new methods for measuring activities and applying an overarching theoretical framework’’ would help to provide a clearer picture as to the causal mechanisms particular to ECA and positive outcomes for secondary school students. Analysis strategy Statistical data were extracted from the 29 studies. Due to the very wide and diverse range of activities examined in these studies, clustering was necessary for analysis purposes (Lipsey and Wilson 2001). In this case, the ECA were clustered into eight groups, as shown in Table 4. Outcomes The outcomes for participation in ECA used in the analysis were: academic achievement (GPA, Maths, English/Reading and Science); retention or school attendance; attitudes towards school; tertiary participation and aspirations to tertiary education; and self-concept. Table 4 Description of activity clusters Activity name Description Sports Any type of individual/team sports activities Academic Club/ Journalism Academic clubs that were subject oriented; journalism/yearbook Performing Arts/ Cheerleading Any type of performing arts e.g. dance, music, theatre, cheerleading etc. Student Council Any type of formal student leadership activities Vocational Club Any types of activities focusing on vocational skills Mentoring Tutoring, mentoring Non Sports Activities Activities that did not include sports yet without any specific details on the type of activity ECA General ECA activities with no details on the types of activities Do extra-curricular activities in schools improve educational outcomes? 601 123
  • 12. Calculating the effect sizes Effect sizes were calculated from each study using formulae suggested by Lipsey and Wilson (2001). The total effect sizes (Cohen’s d) for each outcome were weighted by sample sizes. Where findings did not meet the significance level of p 0.05, an effect size of 0 was imputed as it was assumed that such effects were due to random effects rather than participation in ECA. This conservative approach was taken (only a few non- significant effect sizes were negative) for three reasons: (i) most effect sizes reporting a significance level of p [ 0.05 were small; (ii) non-significant results (with or without data on the effect size) were reported in studies with a large number of participants (n [ 300), so the sample size was likely to have little effect on the significance level; and (iii) some of the reports with non-significant results provided no data on the effect size. Hence, although this conservative approach may converge the effect sizes towards zero, it also limits the possible effects of artefacts on the results, which is important in any study but particularly when there is an attempt to measure causal effects. By taking this very conservative approach it was not possible to estimate the actual effect sizes of ECA, as results using this approach are likely to be biased downward (given that most non-significant effects were positive) (Lipsey and Wilson 2001). However, by minimising the chances of false positive or type 2 error (Abramson 1995) and thus increasing the power of the study, it is believed that this approach is adequate to address the question ‘‘Do extra-curricular activities affect student outcomes?’’ This is particularly important given that critiques of meta- analyses, which use observational data, suggest that uncontrolled confounders may bias the results, particularly if the effects are small (Feinstein 1995; Shapiro 1997). Results The findings from studies on participation in extra-curricular activity and students’ outcomes suggest a positive relationship between participation in school-sponsored activities and achievement (see Table 5). However, it is unclear exactly what it is about these types of activity that causes the learning-related outcomes, or indeed whether there is a causal relationship at all. The analysis of the studies undertaken in this review reinforces this point by demonstrating that while a significant relationship was seen between activities and outcomes in some instances, no evidence of causality was found. Furthermore, in many instances effect sizes were low, indicating that there was no meaningful association at all. Table 5 outlines a summary of the effect sizes, in terms of learning-related outcomes, and indicates that most effect sizes (Cohen’s d) were moderate to low. Meaningful effect sizes were found in only two instances: the effect of general ECA on aspiration to tertiary studies; and the effect of student council participation on GPA. Table 5 shows that effect sizes were exhibited mostly where the outcomes related to grades in general, or specific subjects such as Maths, English/reading and 602 B. Shulruf 123
  • 13. Science. Interestingly, while the effect size for aspiration to tertiary education yielded a moderate effect size of 0.48, no significant effect size was found that related any extra-curricular activity to actual participation in tertiary education. One of the highest effect sizes found was related to participation in student council or student leadership bodies. Participation in student leadership bodies yielded an effect size of 0.47 across three studies on GPA, yet there was no meaningful effect on other outcomes (see Table 5). Participation in academic clubs yielded effect sizes of 0.34–0.37 in English/ Reading, Maths and Science (but negligible for GPA; ES = 0.02). This finding suggests that, although the effect sizes are small, students who participate in academic clubs have higher grade levels. The number of effect sizes measured for each outcome related to academic clubs is relatively high (n = 10–16), which gives higher confidence in the generalisation of this statement (see Table 5). At the same time, however, the reasons for the relationship between academic clubs and higher grades are unclear. It may be speculated that more academically-inclined students choose to participate in academic clubs, but further research is needed to determine causality in the first instance. Table 5 Mean effect sizes by type of activity and outcome Activity cluster Outcome ESa 95% CI N (ES) n (students) ECA General GPA 0.17 -0.31 to 0.66 3 35,866 ECA General Math 0.38 0.16 to 0.60 4 21,644 ECA General English/Reading 0.30 0.11 to 0.49 4 21,644 ECA General Science 0.27 0.11 to 0.43 4 21,644 ECA General Attitudes toward school 0.20 0.11 to 0.29 5 2,904 ECA General Aspiration to Tertiary 0.48 0.16 to 0.79 3 27,596 Sports GPA 0.15 0.07 to 0.23 15 59,804 Sports Math 0.08 -0.04 to 0.20 14 61,032 Sports English/Reading -0.01 -0.07 to 0.05 13 44,727 Sports Retention 0.31 0.09 to 0.52 5 20,960 Sports Aspiration to Tertiary 0.16 -0.10 to 0.42 4 11,993 Sports Self concept 0.04 -0.04 to 0.13 7 37,034 Academic Club/Journalism GPA 0.02 -0.05 to 0.09 10 32,255 Academic Club/Journalism Math 0.36 0.10 to 0.62 16 35,799 Academic Club/Journalism English/Reading 0.37 0.18 to 0.56 16 35,799 Academic Club/Journalism Science 0.34 0.18 to 0.49 13 10,680 Performing Arts/Cheerleading GPA 0.17 0.01 to 0.34 8 16,925 Performing Arts/Cheerleading Math 0.05 -0.02 to 0.13 11 79,430 Performing Arts/Cheerleading English/Reading 0.05 -0.12 to 0.22 11 79,430 Performing Arts/Cheerleading Retention 0.02 -0.19 to 0.23 3 15,541 Students Council GPA 0.47 -0.02 to 0.95 3 7,283 Students Council Math 0.15 0.04 to 0.26 3 13,344 Students Council English/Reading 0.16 0.04 to 0.28 3 13,344 a Effect sizes are weighted for sample size Do extra-curricular activities in schools improve educational outcomes? 603 123
  • 14. Sport was the most commonly investigated extra-curricular activity across the 29 studies. However, the effect of sports activities across the range of learning outcomes was minimal, and ranged between -0.01 and 0.31. While the results do suggest that sports activities could be related to school retention, the effect size (Cohen’s d) of 0.31 is low (Cohen 1977) (Table 5), somewhat limiting the meaningfulness of this finding; although the results tell us that student participation in sports is associated with retention, the correlational evidence does not explain why this might be the case. There were no meaningful associations found for performing arts/cheerleading across the range of outcomes, nor for vocational clubs or mentoring. The effect sizes for performing arts/cheerleading ranged between 0.02 and 0.17 and it is interesting to note that, unlike sports, performing arts did not have any effect on retention. Effect size by school grade level The broader focus of this systematic review is on pathways to higher education; as such it was clearly important to identify the effect size of ECA by the school grade level. Results of the full analysis, effect sizes and their level of significance are shown in Table 6. The most important finding was the relationship between participation in academic clubs and results in core curriculum assessments. Effect sizes for participation in academic clubs at junior and senior high school levels on Maths and English/Reading outcomes were relatively high (ES = 0.86 and 0.69 respectively); for Science the effect size was low-moderate (ES = 0.38). None of the academic clubs that took place in junior schools (J) yielded meaningful effect sizes. These results clearly indicate that participation in academic clubs in ‘Junior and High’ schools is associated with higher grades in Maths, English and Science (ES = 0.86, 0.69, 0.38 respectively), whereas engagement in academic clubs in junior schools is not related to students’ achievements. This conclusion is also supported by the relatively large number of the effect sizes (10) included in this analysis (see Table 6). A different picture can be drawn when looking at the relationship between sports activities and learning-related outcomes. In this case the results show that across junior and high school levels, all effect sizes but one were very small and for some the 95% CI indicated inconclusive results. The exception was sports activities undertaken at a junior school level; this yielded a reasonable (low-moderate) effect size in terms of retention (ES = 0.30, 95% CI 0.02–0.58). Analysis of non-specific general ECA at junior through high school level yielded low to moderate effect sizes for Maths, English and Science outcomes (ES = 0.38, 0.30, 0.27, respectively). At junior school level, non-specific general ECA yielded high effect size for attitudes towards school (ES = 3.65; 95% CI 0.38–6.92) and GPA (ES = 1.8; 95% CI -6.88 to 10.48). However, this finding came from two studies only, so any generalisation should be taken with caution. It is noteworthy that the data available for the meta-analysis did not allow the placement of robust control measures for confounding the effects relating to specific groups’ characteristic or activity types. While this commonly happens when too few 604 B. Shulruf 123
  • 15. effect sizes are identified, some caution is suggested with regard to the interpretation of the mean effect sizes reported above. Effects of data sources As previously mentioned, the studies included for analysis in this review were predominantly based on secondary data sources, a methodology which, in terms of data collection, can lend itself to vulnerability to bias due to limitations around transparency. To examine the relationship between the results of the studies in this review and the methods used by the researchers, the association between the data collection method and the effect sizes was measured. It was found that studies which used secondary data (data taken from existing databases) yielded effect size of 0.15 (95% CI 0.14–0.15), whereas primary data (e.g. questionnaires) yielded higher effect size (ES = 0.27 95% CI 0.26–0.28). Since these differences are significant, it is suggested that any further systematic review/meta-analysis in this area should consider the effects of the data sources. It is particularly important in the use of secondary data as it may mean that the researchers Table 6 Mean effect sizes by school grade level Activity cluster Outcome School levela ESb 95% CI N (ES) Sports Self concept J & H 0.04 -0.04 to 0.13 7 37,034 Sports Retention J 0.30 0.02 to 0.58 4 20,568 Sports English/Reading J -0.01 -0.35 to 0.32 3 35,197 Sports GPA J 0.17 -0.08 to 0.42 6 22,004 Sports GPA J & H 0.14 0.10 to 0.19 7 37,034 Sports Math J 0.12 -0.66 to 0.90 3 35,197 Sports Math J & H 0.03 0.01 to 0.04 9 25,069 Performing Arts/ Cheerleading English/Reading J 0.11 -0.13 to 0.34 6 67,857 Performing Arts/ Cheerleading GPA J 0.18 -0.15 to 0.51 4 15,393 Performing Arts/ Cheerleading Math J 0.06 -0.07 to 0.19 6 67,857 ECA General Attitudes towards school J 3.65 0.38 to 6.92 6 2,904 ECA General Science J & H 0.27 0.11 to 0.43 4 21,644 ECA General English/Reading J & H 0.30 0.11 to 0.49 4 21,644 ECA General GPA J 1.80 -6.88 to 10.48 4 35,866 ECA General Math J & H 0.38 0.16 to 0.60 4 21,644 Academic Club/Journalism Science J & H 0.38 0.20 to 0.55 10 9,530 Academic Club/Journalism English/Reading J & H 0.69 0.38 to 1.00 10 9,530 Academic Club/Journalism GPA J 0.01 -0.07 to 0.09 4 27,839 Academic Club/Journalism Math J & H 0.86 0.41 to 1.31 10 9,530 a J & H Junior and High school; J Junior. b Effect sizes are weighted for sample size Do extra-curricular activities in schools improve educational outcomes? 605 123
  • 16. are working blind, rather than being privy to information that has the potential to elucidate and inform the research. For example, knowing whether a respondent was a participant or programme operator may change the value of the data. Discussion The key issue raised by this critical review and meta-analysis of literature on participation in ECA and the relationship of participation to school-related outcomes is linked to the strength of the evidence supporting the effect of ECA on educational outcomes, and the limitations around the determination of causality. A further, related issue highlighted is the restriction placed on both the analysis and the findings due to the lack of empirical data (e.g. attendance and ethnicity data), and the popular use of data collected from secondary sources. Causality The studies investigated in this review set out to illustrate how participation in ECA is linked to a range of school-related outcomes, including higher academic achievement, engagement and retention in schooling, development of leadership and self-responsibility and future tertiary aspirations. While relationships were determined; there was no evidence of causal effects. Causal effect For a relationship to demonstrate causal effect, the following criteria should be addressed (Abramson 1995, pp. 259–267): a. Is the association between the suspected cause and the effect shown to be statistically significant (at the level determined by the researcher)? b. Is the strength of the association such as correlation between cause and effect high enough and the actual value context-related? c. Is there evidence that the greater the exposure to the cause, the greater the effect? (‘Dose response’ relationship) d. Has the cause occurred before the effect appears (time response)? e. Is the effect related to only one cause (specificity)? (If so, the support for causal effect increases.) f. Does the association show consistency under different conditions? and; g. Is there coherence with current knowledge? (Inconsistency with other evidence weakens the case for causality.) The evidence provided in some of the studies in this review met some of the criteria, but no study met them all. Although some associations were statistically significant (see criterion a), the strength of the association, namely the effect sizes, were small (criterion b). No evidence could identify dose exposure (criterion c), or any ‘Time response’ relationship (criterion d), as there was either no measure for change over time or no indication in these studies that students’ achievement had 606 B. Shulruf 123
  • 17. increased/decreased over the time. It was impossible to indicate level of specificity (criterion e), as no study controlled the associations for other known effects on the outcomes (such as family variables, school experiences etc.). The differences in the effect sizes across different studies – including negative effect sizes – provide little support that the evidence presented met the criterion of consistency (criterion f). Finally, the current knowledge about the impact of ECA on academic achievements is too scarce to suggest any concrete conclusion (criterion g). Some caution should be taken, however, in this causality assessment. As Abramson (1995, p. 261) suggested, ‘‘We can never really ‘prove’ a causal relationship. What we want, however, is ‘reasonable proof’, strong enough to be used as basis for decision and action.’’ In practice this means that the more criteria for causality are met, and the more substantial these criteria are, the more likely it is for a causal effect to be evident. Thus, ‘‘decisions and action’’ should be made on the best evidence available, and the task of providing this evidence should be central to further research in this area. Overall, although some associations could be identified between participation in ECA and a number of students’ outcomes, there was no robust evidence for causal effects relating to these associations. Until causality can be shown, how best to enhance positive school-related outcomes through ECA will remain unclear. Limitations of study data The collection of data from the extra-curricular studies presented the review authors with some difficulties due to the widely varied approaches taken, the multiplicity of outcomes and the varying definitions of ‘‘outcomes’’. For example, studies used a wide range of academic outcome measures, including: GPA, Scholastic Assessment Test (SAT) scores, grades for single core curriculum subjects, course credits, and self-reports/teacher ratings in the United States; and GCSE in the United Kingdom. Studies also focused on a range of different core curriculum areas; some examined English, some Science, some Maths, and others a mixture or all three. There was also an issue with the range and clustering of ECA. There were no data available on activity attendance, and limited data available on ethnicity, both of which restricted the ability to look at trends and patterns. Furthermore, although the data generally originated from longitudinal studies with baseline data, there appeared to be no controls in place which might have enabled a comparison of outcomes by baseline data to have taken place. Use of secondary data Most of the studies used secondary data for their analyses but, although these datasets are large and quite comprehensive, the data stored in these databases were not originally collected to address specific questions about ECA. For example, question 82 from the first National Education Longitudinal Study in 1988 (NELS:88) asks: ‘‘Have you or will you have participated’’ in ECA (see Fig. 1). This question was very likely designed to collect information to answer a specific research question, and not necessarily the one presented in some of the studies reviewed. This question asks about the future, and does not distinguish between Do extra-curricular activities in schools improve educational outcomes? 607 123
  • 18. reports on actual activities taken, or plans to take these in the future; nor does it ask for relevant or adequate information related to the outcomes of ECA. In addition, this question does not distinguish between the level or the type of participation in ECA, and as such it is impossible to determine, for instance, whether the respondent is a participant, a leader or an organiser. Another issue relating to the use of secondary source data, and the NELS:88 in particular, is the lack of depth to the information collected. General comprehensive questionnaires, which ask many questions on a broad range of topics, are designed to capture wide-ranging information rather than collecting in-depth information on specific topics. As 13 studies included in this review used data from NELS:88, and all of them used the items from the first wave as presented above, the validity of the information on participation in ECA is doubtful. Methodological bias Further examination of the approaches used in studies based on secondary source data revealed more methodological flaws. For example, Marsh and Kleitman (2003) used NELS:88 for their study and sampled 12,084 students from the 1988 cohort. However, on closer examination, their study included only 4,250 students (35%) with valid data for all the variables considered in the analysis. Moreover, students who transferred or dropped out of school were excluded from the main analyses. Providing no analysis of missing data and excluding the students who left school raises a critical question about whether or not the effects found in this analysis are only artefacts. Alternatively, the results could be overturned if all the data were included in the analysis. For example, one might assume that there were many students, among the 65% who were excluded from the study, who participated in ECA but left school for various reasons, including poor outcomes. None of the other studies which used NELS:88 revealed any analyses of missing data. In addition, it was found that the data used excluded students who changed schools and those who did not complete questionnaires (or whose parents did not complete the questionnaire). The vulnerability for bias is great as students with low grades are more likely to drop out or change schools and their parents are less likely to be engaged in school life. Excluding these students from the analysis, while it is known that the majority of students are active in ECA (Cooper et al. 1999), is likely to bias the reported results upwards. These possible biases have a major effect on the analyses carried out in the current review, since there were quite a few studies which included subsamples from the NELS:88 project and therefore some participants might have been included in more than one study. For this reason students’ weightings were not equal across studies and, 82. Have you or will you have participated in any of the following school activities during the current school year, either as a member, or as an officer (for example, vice-president, coordinator, team captain)? Fig. 1 Question 82 from NELS of 1988 608 B. Shulruf 123
  • 19. as individuals could not be identified, it was impossible to control this bias using statistical measures. While the conceptual consequences are obvious, this situation provides further support for the conservative analytical approach undertaken in this systematic review. Conclusion The aim of this systematic review was to ascertain what it is about ECA participation that supports positive outcomes and why, and to consider how the common assumptions about the benefits of ECA participation can be validated. The results show associations rather than causation and raise major concerns regarding the validity of some of the data and analyses used in the literature. This leads to the conclusion that the current knowledge on ECA participation does not suggest that extracurricular activities affect student educational outcomes either positively or negatively. It is therefore considered essential that further research be carried out to unravel how participation in ECA contributes to students’ outcomes and why. Such research should investigate aspects of participation including what motivates participation, how and why students participate, and how such participation impacts on their outcomes. Recommendations for further research Lack of evidence for causality strongly suggests that the fundamental question which should be asked is: ‘‘Do extra-curricular activities have any effect on students’ outcomes?’’ It is therefore recommended that further studies try to identify the effects of ECA on students’ outcomes by including measures for causality in their study design. The best measures for causal effects are established by prospective randomised controlled trials (Abramson 1995) and studies incorporating this methodology in this area should be promoted. It is noted however, that undertaking prospective randomised controlled trials is very difficult in social sciences, particularly due to ethical issues. Thus, the second-best option is to apply methods of quasi-experiment with an emphasis on causal effects; that is, collecting data which enable assessment of as many causality criteria as possible. Implementing such measures in the ECA research is important. Schools in most countries spend significant resources on ECA but, as this critical review has found, there is no robust evidence supporting the effectiveness of that expenditure. Thus, research in this area is crucial, particularly if one wishes to improve school effectiveness and student outcomes. References Evaluation studies reviewed are marked* Abramson, H. J. (1995). Survey methods in community medicine. New York: Churchill Livingstone. Do extra-curricular activities in schools improve educational outcomes? 609 123
  • 20. Adelman, C. (1995). The new college course map and transcript files: Changes in course-taking and achievement, 1972–1993. Based on the postsecondary records from two national longitudinal studies. Washington, DC: National Institute on Postsecondary Education, Libraries, and Lifelong Learning. Ainley, J. (1994). Multiple indicators of high school effectiveness. Paper presented at the Annual meeting of the American Educational Research Association, New Orleans. Appelman, M., Gorter, J., Lijesen, M., Onderstal, S., & Venniker, R. (2003). Equal rules or equal opportunities? Demystifying level playing field. Hague: CPB Netherlands Bureau for Economic Policy Analysis. *Barber, B. L., Eccles, J. S., & Stone, M. R. (2001). Whatever happened to the jock, the brain, and the princess? Young adult pathways linked to adolescent activity involvement and social identity. Journal of Adolescent Research, 16(5), 429–455. Barr, H., Hammick, M., Koppel, I., & Reeves, S. (1999). Evaluating interprofessional education: Two systematic reviews for health and social care. British Educational Research Journal, 25(4), 533–544. Baxter, S. G. (1936). Intelligence and the extra-curriculum activities selected in high school and college. The School Review, 44(9), 681–688. Boaz, A., Ashby, D., & Young, K. (2002). Systematic reviews: What have they got to offer evidence based policy and practice? ESRC UK Centre for Evidence Based Policy and Practice, Queen Mary University of London. *Braddock II, J. H., Royster, D. A., Winfield, L. F., & Hawkins, R. (1991). Bouncing back: Sports and academic resilience among African-American males. Education and Urban Society, 24(1), 113–131. *Branch, J. L. (2003). Extracurricular activities and academic achievement. Hattiesburg: The University of Southern Mississippi. *Broh, B. A. (2002). Linking extracurricular programming to academic achievement: Who benefits and why? Sociology of Education, 75(1), 69–95. *Brown, R., & Evans, W. P. (2002). Extracurricular activity and ethnicity: Creating greater school connection among diverse student populations. Urban Education, 37(1), 41–58. *Buoye, A. J. (2004). Capitalizing in the extra curriculum: Participation, peer influence, and academic achievement. Notre Dame, IN: University of Notre Dame. *Chambers, E. A., & Schreiber, J. B. (2004). Girls’ academic achievement: Varying associations of extracurricular activities. Gender and Education, 16(3), 327–346. Cohen, J. (1977). Statistical power analysis for behavioral sciences. New York: Academic Press. Cooper, H., Valentine, J. C., Nye, B., & Lindsay, J. J. (1999). Relationships between five after-school activities and academic achievement. Journal of Educational Psychology, 91(2), 369–378. *Darling, N., Caldwell, L. L., & Smith, R. (2005). Participation in school-based extracurricular activities and adolescent adjustment. Journal of Leisure Research, 37(1), 51–77. *Davalos, D. B., Chavez, E. C., & Guardiola, R. J. (1999). The effects of extracurricular activity, ethnic identification, and perception of school on student dropout rates. Hispanic Journal of Behavioral Sciences, 21(1), 61–77. Davies, P. (2000). The relevance of systematic reviews to educational policy and practice. Oxford Review of Education, 26(3–4), 365–378. *Diaz, J. D. (2005). School attachment among Latino youth in rural Minnesota. Hispanic Journal of Behavioral Sciences, 27(3), 300–318. *Eccles, J. S., & Barber, B. L. (1999). Student council, volunteering, basketball, or marching band: What kind of extracurricular involvement matters? Journal of Adolescent Research, 14(1), 10–43. Eccles, J. S., Barber, B. L., Stone, M., & Hunt, J. (2003). Extracurricular activities and adolescent development. Journal of Social Issues, 59(4), 865–889. Feinstein, A. R. (1995). Meta-analysis: Statistical alchemy for the 21st century. Journal of Clinical Epidemiology, 48(1), 71–79. Feldman, A. F., & Matjasko, J. L. (2005). The role of school-based extracurricular activities in adolescent development: A comprehensive review and future directions. Review of Educational Research, 75(2), 159–210. *Gerber, S. B. (1996). Extracurricular activities and academic achievement. Journal of Research and Development in Education, 30(1), 42–50. *Guest, A., & Schneider, B. (2003). Adolescents’ extracurricular participation in context: The mediating effects of schools, communities, and identity. Sociology of Education, 76(2), 89–109. 610 B. Shulruf 123
  • 21. Holland, A., & Andre, T. (1987). Participation in extracurricular activities in secondary school: What is known, what needs to be known? Review of Educational Research, 57(4), 437–466. Holland, M. N. (1933). Extra-curriculum activities in high schools and intermediate schools in Detroit. The School Review, 41(10), 759–767. *Jordan, W. J. (1999). Black high school students’ participation in school-sponsored sports activities: Effects on school engagement and achievement. Journal of Negro Education, 68(1), 54–71. *Kaufman, J., & Gabler, J. (2004). Cultural capital and the extracurricular activities of girls and boys in the college attainment process. Poetics, 32(2), 145–168. *Keith, T. Z., Diamond-Hallam, C., & Goldenring Fine, J. (2004). Longitudinal effects of in-school and out-of-school homework on high school grades. School Psychology Quarterly, 19(3), 187–211. Kitchenham, B. (2004). Procedures for performing systematic reviews (No. 0400011T.1). Keele, UK: Software Engineering Group, Department of Computer Science, Keele University. Konstantopoulos, S. (2005). Trends of school effects on student achievement: Evidence from NLS:72, HSB:82, and NELS:92. Bonn: The Institute for the Study of Labor. Lewis, C. P. (2004). The relation between extracurricular activities with academic and social competencies in school age children: A meta-analysis. College Station: Texas A&M University. *Lipscomb, S. R. (2007). Secondary school extracurricular involvement and academic achievement: A fixed effects approach Economics of Education Review, 26(4), 463–472. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: SAGE Publications. *Lisella, L. C., & Serwatka, T. S. (1996). Extracurricular participation and academic achievement in minority students in urban schools. Urban Review, 28(1), 63–80. *Mahoney, J. L., Cairns, B. D., & Farmer, T. W. (2003). Promoting interpersonal competence and educational success through extracurricular activity participation. Journal of Educational Psychol- ogy, 95(2), 409–418. *Mahoney, J. L., & Cairns, R. B. (1997). Do extracurricular activities protect against early school dropout? Developmental Psychology, 33(2), 241–253. *Marsh, H. W. (1992). Extracurricular activities: Beneficial extension of the traditional curriculum or subversion of academic goals? Journal of Educational Psychology, 84(4), 553–562. *Marsh, H. W., & Kleitman, S. (2003). School athletic participation: Mostly gain with little pain. Journal of Sport & Exercise Psychology, 25(2), 205–228. *McCarthy, K. J. (2000). The effects of student activity participation, gender, ethnicity, and socio- economic level on high school student grade point averages and attendance. Boulder: University of Colorado. *McNeal, R. B. (1995). Extracurricular activities and high school dropouts. Sociology of Education, 68(1), 62–80. *Melnick, M. J., Sabo, D. F., & Vanfossen, B. (1992). Educational effects of interscholastic athletic participation on African-American and Hispanic youth. Adolescence, 27(106), 295–308. NCES. (2006). NELS:88/2000. Fourth follow-up: An overview. Retrieved February 24, 2006, from http://nces.ed.gov/surveys/nels88/. Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004). Realist synthesis: An introduction. Manchester: ESRC Research Methods Programme, University of Manchester. *Power, A. R. (1999). Getting involved and getting ahead: Extracurricular participation and the educational attainment process. Notre Dame, IN: University of Notre Dame. Robinson, J., & Etheridge, B. (1995). The 1994–95 North Carolina State testing results. Multiple-choice end-of-grade and end-of-course Tests including English II essay. Raleigh: North Carolina State Department of Public Instruction. Shapiro, S. (1997). Is meta-analysis a valid approach to the evaluation of small effects in observational studies? Journal of Clinical Epidemiology, 50(3), 223–229. *Silliker, S. A., & Quirk, J. T. (1997). The effect of extracurricular activity participation on the academic performance of male and female high school students. School Counselor, 44(4), 288–293. *Slicker, E. K., & Palmer, D. J. (1993). Mentoring at-risk high school students: Evaluation of a school- based program. School Counselor, 40(5), 327–334. Suri, H., & Clarke, D. (1999). Revisiting methods of literature synthesis. Melbourne: University of Melbourne. Taylor, J. L., & Chiogioji, E. N. (1988). The Holland and Andre study on extracurricular activities: Imbalanced and incomplete. Review of Educational Research, 58(1), 99–105. Do extra-curricular activities in schools improve educational outcomes? 611 123
  • 22. Thomas, W. B., & Moran, K. J. (1991). The stratification of school knowledge through extracurricular activities in an urban high-school. Urban Education, 26(3), 285–300. Torgerson, C. J. (2006). Publication bias: The Achilles’ heel of systematic reviews? British Journal of Educational Studies, 54(1), 89–102. Valentine, J. C., Cooper, B., Bettencourt, A., & DuBois, D. L. (2002). Out-of-school activities and academic achievement: The mediating role of self-beliefs. Educational Psychologist, 37(4), 245–256. The author Dr. Boaz Shulruf is the Deputy Head of the Centre of Medical and Health Sciences Education at the Faculty of Medical and Health Sciences, University of Auckland. His main research interest is in methodology of research in education, cross-cultural psychology, social sciences and health. His recent publications include articles on student pathways in the secondary and tertiary education; methodology of longitudinal studies; and collectivism and individualism. 612 B. Shulruf 123
  • 23. Copyright of International Review of Education / Internationale Zeitschrift für Erziehungswissenschaft is the property of Springer Science & Business Media B.V. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.