Education Outcomes and Contextual Data: the Potential of International Surveys like PISA


Published on


Presented by Simone Bloem, OECD Directorate for Education, Programme for Co-operation with Non-member economies at the Regional Conference on Fighting Corruption in Education in Eastern Europe and Central Asia, 24-25 November 2011, Bratislava, Slovakia

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Education Outcomes and Contextual Data: the Potential of International Surveys like PISA

  1. 1. Education outcomes andcontextual data: the potential ofinternational surveys like PISA Simone Bloem OECD Directorate for Education Programme for Co-operation with Non-member economies
  2. 2. Outline• Overview of international surveys: PISA TIMMS PIRLS• Relevance of PISA for assessing integrity of education systems• School system characteristics that are positively associated with performance, equity and integrity• Next steps 2
  3. 3. International Surveys • OECD’s Programme forPISA International Student Assessment • IAE’s Trends in InternationalTIMMS Mathematics and Science Study • IAE’s Progress in InternationalPIRLS Reading Literacy Study 3
  4. 4. OECD: PISATriennial survey of key competencies of 15-year-old studentsMain subjects tested: reading (2000, 2009),mathematics (2003) and science (2006)Tool to compare the efficiency and equity ofcountries’ education systems in an internationalperspective
  5. 5. A map of PISA countries and economies 5
  6. 6. IAE: TIMMSAssessment of mathematics andscience competencies of 4th- and 8th-grade studentsSurvey conducted every four years:1995, 1999, 2003, and 2007Participation of more than 60countries, diverse in terms ofgeographical location and economicdevelopment
  7. 7. IAE: PIRLSAssessment of students’ reading literacy at theend of primary school, in some countriesstudents are assessed in fifth and sixth gradePIRLS is conducted every five years : 2001, 2006and 201155 countries from all around the world haveparticipated in PIRLS 2011PIRLS 2011 has been extended to includeprePIRLS —less difficult and designed to testbasic reading skills that are a prerequisite forPIRLS
  8. 8. Reading Mathematics Science Mean score Mean score Mean score Mean S.E. Mean S.E. Mean S.E.OECDEstonia 501 (2.6) 512 (2.6) 528 (2.7)Poland 500 (2.6) 495 (2.8) 508 (2.4)Hungary 494 (3.2) 490 (3.5) 503 (3.1)OECD average 493 (0.5) 496 (0.5) 501 (0.5)Latvia 484 (3.0) 482 (3.1) 494 (3.1)Slovenia 483 (1.0) 501 (1.2) 512 (1.1)Czech Republic 478 (2.9) 493 (2.8) 500 (3.0)Slovak Republic 477 (2.5) 497 (3.1) 490 (3.0)Croatia 476 (2.9) 460 (3.1) 486 (2.8)Lithuania 468 (2.4) 477 (2.6) 491 (2.9)Russian Federation 459 (3.3) 468 (3.3) 478 (3.3)Serbia 442 (2.4) 442 (2.9) 443 (2.4)Bulgaria 429 (6.7) 428 (5.9) 439 (5.9)Romania 424 (4.1) 427 (3.4) 428 (3.4)Montenegro 408 (1.7) 403 (2.0) 401 (2.0)Albania 385 (4.0) 377 (4.0) 391 (3.9)
  9. 9. Background dataStudents background questionnaire • socio-economic background, learning habits, attitudes towards reading, involvement and motivationSchool principals questionnaire • demographic school characteristics, school policies, quality of learning environmentParents questionnaire • PISA for selected countries (optional), PIRLS • home learning environment, parental support, school choiceTeachers Questionnaire • For TIMMS and PIRLS only • Professional background, teaching methods, training and professional developmentCurriculum questionnaires for National Research Coordinators • TIMMS and PIRLS onlyPIRLS/TIMMS 2011 Encyclopedia • provides a profile of each country’s education system
  10. 10. International surveys deliver… International indicators and benchmarks Analysis in theAnalysis in the national and international regional context context Evidence- based, effective education policies 10
  11. 11. Integrity and PISA• Starting point to tackle corruptive practices is to evaluate the integrity of an educational system• We are assuming that systems with high level of integrity are in Successful general those systems school system that perform well – well in terms of student • In PISA, successful school achievement and systems are defined as those PISA providing equal that perform above the OECD opportunities. average in reading and in which students’ socio-economic • Allows identifying background has a smaller school characteristics impact on reading performance that are shared by Integrity than it is the case in a typical successful school OECD country. systems • The term ―successful school • PISA is used as a first system‖ refers o two distinct point of reference in areas: achievement and order to assign equity education policy areas of integrity
  12. 12. School characteristics related to performance and/or equityLow levels ofdifferentiation Grade repetition Student transference Ability grouping
  13. 13. Low differentiation of the school system Differentiation : Access • Unequal access to schools and classrooms creates demand situations for scarce places which may lead to corruption, in particular when perceived quality of schools differs greatly
  14. 14. School characteristics related to performance and/or equityLow levels of Accountabilitydifferentiation Grade repetition The existence of standards-based external examinations Student transference And how they are used Ability grouping Monitoring of lessons by inspectors
  15. 15. AccountabilityStandards-based external examinations• Educational achievement is made observable and transparent• Facilitate the monitoring of performance of students, teachers, and schools• Can be considered as best practice to avoid opportunistic behaviour by teachers and principals • Teachers cannot “get away” by skipping content areas or employ very easy tests to have a positive teacher performance evaluation • Since teachers do not know exam questions and areas covered in the test beforehand, incentives for corruption are nonexistent, e.g. selling exam questions in advanceExternal final exams• Improve the signal effect of school leave certificates and final examinations marks which are of particularly importance for higher educational institutions or future employers and may reduce corruption, e.g. fraud at university admission or favouritism, political affiliation in employment
  16. 16. School characteristics related to performance and/or equityLow levels of School Accountabilitydifferentiation autonomy Grade repetition The existence of Autonomy to standards-based make decisions on external curricula and examinations assessments Student transference And how they are used Ability grouping Monitoring of lessons by inspectors
  17. 17. School autonomySchool autonomy• School autonomy has a positive effect on student achievement, but only when appropriate accountability practices are in place• Schools with more autonomy may have more incentives to opportunistic behaviour, in particular when there is a high degree of information asymmetry about school behaviour, e.g. parents, governing boards or central planning authorities have insufficient information on what is happing in the school• “Without external exams, the advantage of superior local knowledge must weighed against the disadvantage of opportunistic behaviour, and the net effect of school autonomy depends on the relative size of these two partial effects.” Wößmann et al. (2008: 34)
  18. 18. School characteristics related to performance and/or equityLow levels of School Spending on Accountabilitydifferentiation autonomy education Grade The existence of Teacher salaries repetition Autonomy to standards-based make decisions external on curricula and examinations assessments Student Minimum resource transference requirements And how they are used Ability grouping Monitoring of lessons by inspectors
  19. 19. Spending on educationHigh teacher salaries• Low teacher salaries are considered as one reason of corruption• Reflect the social status of the teaching profession in a society• Suggests commitment to the rules and guidelines of teaching in a country and satisfaction of profession• Reduced incentives for corrupt and unethical behaviour, e.g. private tutoring or absenteeismAvailability of sufficient human and material resources• The eventuality of corruptive behaviour is greater in schools that lack material, e.g. teachers might feel compelled to ask for money to provide students with learning materials, such as textbooks and laboratory equipment or principals may illegally rent out school facilities• Great inequity between schools in terms of resources may lead to high demand for scarce places in ―better‖ schools. This demand may lead to corruption occurrence
  20. 20. School characteristics related to performance and/or equityLow levels of School Spending on Learning Accountabilitydifferentiation autonomy education environment Grade Autonomy to Teacher Disciplinary repetition The existence of salaries climate standards- make based external decisions on examinations curricula and assessments Minimum Student and Student resource teacher transference requirements behaviour And how they are used Teacher- Ability student grouping relations Monitoring of lessons by inspectors
  21. 21. Learning environmentQuality of learningenvironment• High quality of the learning environment, approximated by indicators such as good teacher-student relations, positive student and teacher behaviour and good disciplinary climate, are important to ensure a high level of moral and commitment in schools and classrooms.• This can be considered as crucial elements of an intact school system that prevent various corruptive or unethical practices.
  22. 22. Next stepsPISA and corruption items• Integration of corruption-related items in the national PISA students’, parents’ and/or school questionnaire, e.g. questions related to private tutoring, gifts to teachers, cheating • Raise awareness of the problem of corruption in the education sector on an international level • Open up new opportunities to research corruption empirically • Make corruption in education empirically measurable and allow to relate corruption to educational outcomes
  23. 23. Thank you for yourattention!