Education outcomes and contextual data, the potential of international surveys like PISA
Upcoming SlideShare
Loading in...5
×
 

Education outcomes and contextual data, the potential of international surveys like PISA

on

  • 822 views

Regional Conference on Fighting Corruption in Education in Eastern Europe and Central Asia, 24-25 November, Bratislava

Regional Conference on Fighting Corruption in Education in Eastern Europe and Central Asia, 24-25 November, Bratislava
Simone Bloem

Statistics

Views

Total Views
822
Views on SlideShare
519
Embed Views
303

Actions

Likes
0
Downloads
5
Comments
0

4 Embeds 303

http://unjobs.org 299
http://www.unjobs.org 2
http://a0.twimg.com 1
http://translate.googleusercontent.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • The presentation will be structured as follows. I will give you a short overview of international surveys, notably PISA, TIMMS and PIRLS. I will then quickly go through contextual data collected in those surveys which allow to assess how national educational policies are related to performance and equity In a second part , I will present the relevance of PISA for assessing integrity of education systems, notably the role of PISA in the methodological assessment framework of OECD’s INTES project. I will then present the school system characteristics that are positively associated with performance and why these characteristics may also related to a high level of integrity. I will conclude the presentation with prospects of how international surveys such as PISA could be used to measure corruption
  • There are three large international surveys on students’ achievement: PISA TIMMS and PIRLS. PISA is an OECD project, TIMMS and PIRLS are conducted by the IAE = The International Association for the Evaluation of Educational Achievement, an international association
  • PISA is the largest and most rigorous survey on students’ performance . It is conducted every three years and measures key competencies of 15-year-old students. In contrast to other surveys, the PISA test is curriculum-free as it is constructed independently of national curriculums. Instead , PISA assesses the extent to which students near the end of compulsory education have acquired some of the knowledge and skills that are considered essential for full participation in modern societies. PISA seeks to assess not merely whether students can reproduce knowledge, but also to examine how well they can extrapolate from what they have learned and apply it in unfamiliar settings, both in and outside of school. Started in 2000, PISA has now completed its fourth round of surveys. While all three subjects are tested at each round, one subject is tested more extensively in each round. In the last survey in 2009 it was reading. Due t o extensive background data, which I will discuss later on, PISA is a tool to compare the efficiency and equity of countries’ education systems in an international perspective .
  • The number of countries that participate in the PISA survey has grown steadily since 2000 to more than 70 countries in the last round 2009. Besides all 35 OECD member countries, shown in grey on the map, there is a growing number of Non-member countries participating, the countries marked in blue. However, as you can see, there is only one country from the African Continent. Central and Eastern Europe are almost all participating, as well as many Asian countries.
  • Let me quickly introduce TIMMS, in full name: Trends in International Mathematics and Science Study (TIMSS). It provides data on the mathematics and science achievement of 4th- and 8th-grade students. TIMSS has started in 1995 and is conducted every four years. In its last survey in 2011, more than 60 countries participated in TIMSS. Countries are in various stages of development and cover diverse geographic locations, including Eastern Europe, Africa and Asia.
  • PIRLS -Progress in International Reading Literacy Study (PIRLS) is as TIMMS coordinated by the International Association for the Evaluation of Educational Achievement (IEA). PIRLS is an international comparative study of the reading literacy at the end of primary school, in some countries students are also assessed in fight and sixth grade. PIRLS was first administered in 2001 and included 35 countries, and was administered again in 2006 to students in 45 education systems. Another round has been or is still conducted in 2011 with approximately 55 countries from all around the world. PIRLS 2011 has been extended to include prePIRLS —less difficult and designed to test basic reading skills that are a prerequisite for PIRLS Also PISA is elaboriting an easier version of the PISA tests for low performing countries.
  • But let me know come to the question you’ve probably all in mind: what do international surveys have to do with corruption or rather anti-corruption policies and measures in education? Let me explain to you where the we consider the linkage between international surveys, notably PISA and integrity assessments. … explain three points
  • Point 1 school systems in which all students, regardless of their background, are offered similar opportunities to learn; socio-economically advantaged and disadvantaged students attend the same schools; and students rarely repeat grades or are transferred out of schools because of behavioural problems, low academic achievement or special learning needs – are more likely to perform above the OECD average and show below-average socio-economic inequalities.
  • In the following part I will establish causal links between integrity and those system characteristics which have been identified as being related to successful school systems. Low level of differentiation, that means all students learn together, late selection and specialisation etc., is crucial to provide all students with equal access to educational services. We consider unequal access to schools and classrooms as a main source of demand situations for scarce places while creating corruption risks. Thereby, if the quality of schools differs greatly, demand and related corruption risk will be further increased. Secondly, transferring policies may open up incentives for opportunistic behaviour for teachers and principals as they can put pressure on students and parents. It is also more likely that principals engage in illegal agreements with principals of other schools if transferring policies are common practice.
  • Point 2 A characteristic that can be considered as an accountability measure are standards-based external examinations. Contrary to teacher developed tests, they define performance relative to an external standard, not relative to other students in the classroom or school. Students in school systems that use standards-based external examinations perform, on average across OECD countries, 16 points higher than students in school systems that do not use these examinations. It s not only mere existence of standardised tests but also how they are used. There is a positive relationship between schools that post achievement data publicly and schools’ performance. Schools that post achievement data publicly tend to perform better than schools whose achievement data is not made publicly available. With PISA 2003 Data – why this box is in a different colour – a positive relationship between the share of students in schools where teachers’ lessons are monitored by the school principal or other senior staff could be found (Wößmann et al. 2008, PISA 2003 data)
  • Standard-based external examinations can be considered as accountability policy as such exams make educational achievement observable and transparent. Moreover, it facilitates the monitoring of performance of students, teachers and schools. External examinations can thus be considered as best practice to avoid opportunistic behaviour by teachers and principals. Firstly, teachers cannot get away by skipping content areas or employ very easy tests to give good marks to students in order to have a positive performance evaluation. Secondly, since teachers do not know exam questions and areas covered in the test beforehand, incentives for corruption are non-existent. For example, standardised external exams hinder the sell of exam questions or private tutoring in exchange for additional information on what is covered in the test External final exams at the end of compulsory secondary education moreover improve the signal effect of school leave certificates and final examination marks which are of particularly importance for higher education institutions or future employers. Final exit exams may reduce corruption, for example fraud in university admission as universities or faculties have less influence on admission criteria. Furthermore, better signal effects of school leaving certificates may reduce unethical behaviour in employment.
  • Point 3 The results from PISA 2009 also show that those school systems that grant individual schools authority to make decisions about curricula and assessments while limiting school competition are more likely to be performing above the OECD average and show below-average socio-economic inequalities. Many school systems with high average performance but comparatively large socio-economic inequalities tend to allow higher levels of school competition. Results from PISA also suggest that giving parents and students a choice of schools does not relate positively to equity in education if their choice is constrained by financial or logistical considerations, such as additional tuition fees or transportation to and from schools.
  • Guiding assumption is that local knowledge lead can make the provision of educational services more efficient since local agents know students’ demands better and can thus act more appropriate on their demands than central planning authorities. However, schools that are entitled to greater autonomy might have more incentives to opportunistic behaviour. This concerns for example, decisions concerning the financial position or workload which schools fulfil. In terms of economic theoretical models, incentives for opportunistic behaviour occur whenever there is a high degree of information asymmetry about school behaviour, e.g. parents, governing boards or central planning authorities have insufficient information on what is happing in the school. Without appropriate accountability measures in place it might be rational for the school decision-makers to favour their own interests over student achievement. Providing greater school autonomy should thus always be bound to accountability practices. Results from PISA provide clear evidence that school autonomy has a positive effect on student achievement, but only when appropriate accountability practices are in place. (read citation)
  • Point 4 School systems that show comparatively high levels of spending by educational institutions and that prioritise teachers’ salaries over class size tend to show better performance. Among systems with comparatively high levels of spending on education which prioritise small class size, performance patterns are mixed. There are also a number of lower-performing systems with similar spending choices. PISA also provides evidence that schools require a certain level of resources to achieve well. Yet, there is no such relationship like “The more resources, the better performance”. Rather one can say, schools that do not possess a minimum level of resources achieving high performance is very low – very often, the poorest schools. Thus, it is important, that all schools, poor and rich schools alike, receive sufficient resources. Point 5 Results from PISA 2009 show that, in general, students perform better in schools with more disciplined classrooms, partly because such schools tend to have more students from advantaged socio-economic backgrounds, who generally perform better, partly because the favourable socio-economic background of students relates to a climate that is conducive to learning, and partly for reasons unrelated to socio-economic factors. Results from PISA 2009 also show that even though the learning environment in schools and classrooms is partially shaped by the resources, policies and practices of the systems and schools, disciplined classrooms themselves tend to go hand in hand with higher performance. BUT : However, the fact that such characteristics are more likely to be found among successful school systems does not mean that they are necessary or sufficient for success. Not all successful school systems share the same organisational characteristics, and not all school systems that are organised in this way achieve high levels of performance and a moderate impact of socioeconomic background on student performance.
  • In anti-corruption literature in education low teacher salaries are time and again considered as a reason for corruption. Yet, this has been particularly relevant if teacher salaries are below minimum living wage, or paid irregularly or not at all. There is good reason that this holds true in a more general context. Teacher salaries reflect the social status of the teaching profession in a society, whereby higher salaries indicate higher social status. High teacher salaries suggest commitment to the rules and guidelines of teaching in a country and satisfaction of profession. Social status, commitment to rules and guidelines and job satisfaction can be considered as important factors to ensure integrity and to reduce incentives for corruptive or unethical behaviour, such as giving private tutoring lessons due to money constraints, or absenteeism because teachers may not feel committed to what they are doing. Having sufficient human and material resources at school is n ecessary to render possible high student achievement and equal opportunities for all student. The eventuality of corruptive behaviour is greater in schools that lack material and causes serious threats to integrity, e.g. teachers might feel compelled to ask for money to provide students with learning materials, such as textbooks and laboratory equipment. Schools without sufficient resources may try to find solutions to compensate their lack, for example may they illegally rent out their school facilities . Another very important point in this context. Great inequity between schools in terms of resources may lead to high demand for scarce places in “better” schools. This demand may lead to corruption occurrence.
  • Point 5 Results from PISA 2009 show that, in general, students perform better in schools with more disciplined classrooms, partly because such schools tend to have more students from advantaged socio-economic backgrounds, who generally perform better, partly because the favourable socio-economic background of students relates to a climate that is conducive to learning, and partly for reasons unrelated to socio-economic factors. Results from PISA 2009 also show that even though the learning environment in schools and classrooms is partially shaped by the resources, policies and practices of the systems and schools, disciplined classrooms themselves tend to go hand in hand with higher performance. BUT : However, the fact that such characteristics are more likely to be found among successful school systems does not mean that they are necessary or sufficient for success. Not all successful school systems share the same organisational characteristics, and not all school systems that are organised in this way achieve high levels of performance and a moderate impact of socioeconomic background on student performance.
  • Why is the learning environment an important area of system integrity. High quality of the learning environment approximated by indicators such as good teacher-student relations, positive student and teacher behaviour and good disciplinary climate, are important elements to ensure a high level of moral and commitment in schools and classrooms. High levels of moral and commitment in turn, are crucial elements to ensure integrity and to prevent various corruptive practices from all stakeholders, students, teachers, and principals.
  • Before I come to the end of the presentation, let me share with you our future propositions. We would welcome the integration of corruption related items in the national PISA questionnaire. It is hereby feasible to include corruption related items or questions in the students, parents or school questionnaire, for example possible questions for parents where spending on their child’s education, private tutoring, or questions that approximate demand for places. Schools may give information about the availability of codes of conducts for teachers, specific practices in place to avoid unethical behaviours from teachers or the like. This would break new grounds in education corruption research. Not only would it raise awareness of the problem of corruption in the education sector on an international level, but it would open up new opportunities to research corruption. These new background variables would make corruption empirically measurable and would allow to relate corruption to educational outcomes – something that has not been done before.

Education outcomes and contextual data, the potential of international surveys like PISA Education outcomes and contextual data, the potential of international surveys like PISA Presentation Transcript

  • Education outcomes and contextual data: the potential of international surveys like PISA Simone Bloem OECD Directorate for Education Programme for Co-operation with Non-member economies
  • Outline
    • Overview of international surveys:
    • PISA TIMMS PIRLS
    • Relevance of PISA for assessing integrity of education systems
    • School system characteristics that are positively associated with performance, equity and integrity
    • Next steps
  • International Surveys
  • OECD: PISA
  • A map of PISA countries and economies
  • IAE: TIMMS
  • IAE: PIRLS
  •   Reading Mathematics Science Mean score Mean score Mean score Mean S.E. Mean S.E. Mean S.E. OECD           Estonia 501 (2.6) 512 (2.6) 528 (2.7) Poland 500 (2.6) 495 (2.8) 508 (2.4) Hungary 494 (3.2) 490 (3.5) 503 (3.1) OECD average 493 (0.5) 496 (0.5) 501 (0.5) Latvia 484 (3.0) 482 (3.1) 494 (3.1) Slovenia 483 (1.0) 501 (1.2) 512 (1.1) Czech Republic 478 (2.9) 493 (2.8) 500 (3.0) Slovak Republic 477 (2.5) 497 (3.1) 490 (3.0) Croatia 476 (2.9) 460 (3.1) 486 (2.8) Lithuania 468 (2.4) 477 (2.6) 491 (2.9) Russian Federation 459 (3.3) 468 (3.3) 478 (3.3) Serbia 442 (2.4) 442 (2.9) 443 (2.4) Bulgaria 429 (6.7) 428 (5.9) 439 (5.9) Romania 424 (4.1) 427 (3.4) 428 (3.4) Montenegro 408 (1.7) 403 (2.0) 401 (2.0) Albania 385 (4.0) 377 (4.0) 391 (3.9)
  • Background data
  • International surveys deliver…
  • Integrity and PISA
  • School characteristics related to performance and/or equity
  • Low differentiation of the school system
  • School characteristics related to performance and/or equity
  • Accountability
  • School characteristics related to performance and/or equity
  • School autonomy
  • School characteristics related to performance and/or equity
  • Spending on education
  • School characteristics related to performance and/or equity
  • Learning environment
  • Next steps
  • Thank you for your attention!