Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Perfromance Information in the Education Sector by Paulo Santiago


Published on

Presentation by Paulo Santiago at the 10th annual meeting of the Senior Budget Officials Performance and Results Network held on 24-25 November 2014. Find more information at

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Perfromance Information in the Education Sector by Paulo Santiago

  1. 1. Performance Information in the Education Sector Session 4a – Performance Informed Budgeting in Practice using a Sectoral Perspective 10th Annual Meeting of the OECD Senior Budget Officials Performance and Results Network Paris, 24-25 November, 2014 From Education System Evaluation to Funding in Education Paulo Santiago, Senior Analyst Directorate for Education and Skills
  2. 2. Outline of Presentation 1. Expenditure in School Education 2. Education System Evaluation: Generating Performance Information 3. The OECD School Resources Review 4. Funding of School Education
  3. 3. 1. Expenditure in School Education
  4. 4. In 2011, OECD countries spent an average of 3.9% of their GDP on primary, secondary and post-secondary non-tertiary education Expenditure on educational institutions as a percentage of GDP (2011). From public and private sources, by level of education and source of funds 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 New Zealand Argentina Iceland United Kingdom Ireland Denmark Belgium Israel Colombia Korea Australia Finland Switzerland Netherlands Canada Mexico France Sweden Slovenia United States Chile Portugal Austria EU 21 average Poland Estonia Luxembourg Spain Germany Italy Latvia Japan Czech Republic Slovak Republic Russian… Norway Brazil % of GDP Primary, secondary and post-secondary non-tertiary education Public expenditure on education institutions Private expenditure on education institutions OECD average (total expenditure)
  5. 5. In 2011, 13% of total public spending was devoted to education Total public expenditure on education as a percentage of total public expenditure (1995, 2005, 2011) 0 5 10 15 20 25 New Zealand Mexico Brazil Korea Switzerland Iceland Denmark Norway Australia Israel Estonia United States Canada Sweden Ireland OECD average Belgium Finland EU21 average United Kingdom Netherlands Austria Poland Slovenia Germany Russian Federation Portugal Slovak Republic Spain Czech Republic France Hungary Japan Italy % of total public expenditure 2011 2005 1995
  6. 6. Source: PISA 2012 Results: What makes schools successful? Resources, policies and practices, Volume IV, Figure IV.1.8. Slovak Republic Czech Republic Estonia Israel Poland Korea Portugal New Zealand Canada Germany Spain France Italy Singapore Finland Japan Slovenia Ireland Iceland Netherlands Sweden Belgium United Kingdom Australia Denmark United States Austria Norway Switzerland Luxembourg Viet Nam Jordan Peru Thailand Malaysia Uruguay Turkey Colombia Tunisia Mexico Montenegro Brazil Bulgaria Chile Croatia Lithuania Latvia Hungary Shanghai-China R² = 0.01 R² = 0.37 300 350 400 450 500 550 600 650 0 20 000 40 000 60 000 80 000 100 000 120 000 140 000 160 000 180 000 200 000 Mathematics performance (score points) Average cumulative spending per student from the age of 6 to 15 (USD, PPPs) Above a certain level of investment, the key factor is how to spend available funding most effectively Student performance and average spending per student
  7. 7. 2. Education System Evaluation: Generating Performance Information
  8. 8. OECD educational policy reviews OECD Reviews of Evaluation and Assessment in Education (Schools) Highlights the importance of context. Different systems are at different stages in developing an evaluation and assessment framework. Expansion of evaluation in school systems Greater reliance on educational standards Increased importance of measurement and indicators Accountability purpose is gaining importance Building capacity for Evaluation and Assessment (E&A) Aligning goals of education system with E&A Going beyond measurement Effectively conceiving accountability Main trends Policy priorities
  9. 9. Education System Evaluation: Generating Performance Information • Drivers: – The rising importance of education in a global world – The growing imperative of an efficient use of public resources – Greater decentralisation and school autonomy – Greater accountability in the public sector – including in budgetary procedures – The growing importance of evidence-based policy • Purposes: – To monitor: • Student outcomes at a given point in time (differences among different regions within the education system and given student groups) and changes in student outcomes over time • Broader outcomes of education systems (e.g. labour market outcomes; social outcomes) • The impact of given policy initiatives or educational programmes • Demographic, administrative and contextual data which are useful to explain the outcomes of the education system – To generate and feedback information for different agents in the education system; – To use the generated information for analysis, development and implementation of policies.
  10. 10. Education System Evaluation: Reference standards – National education goals and objectives, e.g. • Provide high-quality education to students; promote national values and civic responsibilities; develop skills in the economy. – Specific priorities: e.g. improve equity; goals for specific groups – References used in national assessments • National curriculum goals; National learning progressions; National standards; National curriculum goals and standards – Specific targets set to be achieved over a certain timeline • Also supra-national, e.g. EU benchmarks
  11. 11. Education System Evaluation: Specific targets Examples of targets: – Mexico (2007-2012): education system evaluation framed by Education Sector Programme with 6 policy objectives (e.g. promotion of ICT in education) and 41 indicators (each with a target and the respective measure) – Northern Ireland: Programme for Government 2011-15 includes high-level targets for the performance of the education system by 2015 (e.g. 66% of young people achieve at least 5 General Certificates in Secondary Education with a mark of A to C in mathematics, English and three other subjects) – Supra-national, e.g. EU benchmarks to be achieved by 2020 • At least 95% of children between the age of 4 and the age for starting primary education should participate in early childhood education • The share of 15-year-olds with insufficient abilities in reading, mathematics and science should be less than 15% (measure by PISA) • The share of early leavers from education and training should be less than 10% • The share of 30-34 year-olds with tertiary educational attainment should be at least 40%
  12. 12. Education System Evaluation: Procedures • Instruments: – Indicator frameworks; – Tools to monitor student outcomes (national assessments; longitudinal research and surveys; international assessments); – Qualitative reviews of particular aspects of the school system (ad hoc reviews; evaluative information generated via external school system reviews); – Stakeholder surveys; and – The evaluation of specific programmes and policies
  13. 13. Education System Evaluation: Prominence of international student assessments • The profile of the results from international student assessments has been significantly raised in national policy discussions – Perceived as indicators of future economic competitiveness – Highlighted the importance of monitoring student outcomes – Aspirational targets for performance are established (e.g. in PISA) • A number of countries has established PISA targets – Denmark (2010): Danish students to be in the top five countries as judged in international assessments. – Australia (National Plan for School Improvement, 2012): to be among the top five school systems in the world by 2025 in mathematics, science and reading achievement.
  14. 14. Student performance and equity Peru Chile Bulgaria Hungary Slovak Republic Portugal Luxembourg France Uruguay New Zealand Chinese Taipei Belgium Costa Rica Romania Israel Germany Indonesia Colombia Tunisia Argentina Brazil Malaysia Turkey Greece Lithuania Latvia Russian Fed. Spain UK Czech Republic Denmark Slovenia Ireland Austria Viet Nam Switzerland Singapore Shanghai-China Poland United States Croatia Netherlands Montenegro Serbia Hong Kong-China Estonia Finland Thailand Japan Sweden Australia Canada Jordan Macao-China U.A.E. Kazakhstan Iceland Qatar Norway Mexico Liechtenstein Korea Italy 300 350 400 450 500 550 600 650 30 25 20 15 10 5 0 OECD average OECD average Percentage of variance in performance explained by ESCS (r-squared x 100) Mean mathematics performance Below average mathematics performance Below average impact of socio-economic background Above average mathematics performance Below average impact of socio-economic background Below average mathematics performance Above average impact of socio-economic background Above average mathematics performance Above average impact of socio-economic background Relationship between mathematics performance and variation in performance explained by students’ socio-economic status
  15. 15. Education System Evaluation - Example Australia References and standards (National level) • Council of Australian Governments (COAG) National Productivity Agenda – Includes a set of aspirations, outcomes, progress measures, and policy directions for education • Melbourne Declaration on Educational Goals for Young Australians – Articulates future directions and aspirations for Australian schooling • Australian Curriculum • Priority areas / Education targets [e.g. Increased proportion of young Australians attaining secondary education; halve the gap for indigenous students within a decade] Methods and instruments (National level) • National Assessment Program – NAPLAN – Literacy and Numeracy: full cohort tests in reading, writing, spelling, grammar and punctuation, and numeracy at Year levels 3, 5, 7 and 9. – Sample assessments: Cyclical sample surveys in science, ICT, civics and citizenship in Years 6 and 10. – International assessments: PISA, TIMSS, PIRLS • Measurement Framework for National Key Performance Measures • Independent reviews
  16. 16. OECD Indicators of Education Systems (INES) – Education at a Glance The INES organising framework includes three major policy perspectives: • quality of educational outcomes and educational provision; • equality of educational outcomes and equity in educational opportunities; and • adequacy, effectiveness and efficiency of resource management. Comparative information for different levels of education on: Student enrolment, entrance and graduation rates Educational personnel Educational finance Comparative information: Teaching and learning environments (TALIS 2008; PISA surveys) Student outcomes at age 15 Reading, Mathematics and Science (PISA 2000, 2003, 2006, 2009, 2012….) Survey of Adult Skills (16-65) (Literacy, Problem-Solving, Numeracy, Skills-Use) (PIAAC)
  17. 17. Education System Evaluation: Policy Options Governance: Being systematic and strategic for better informed policy making • Ensure a broad concept of education system evaluation within the E&A framework • Ensure policy making is informed by high-quality measures, but not driven by their availability • Situate education system evaluation in the broader context of public sector performance requirements Procedures: Developing an approach to learn from a broad evidence base • Develop a national education indicator framework • Design a national strategy to monitor student learning standards • Ensure the collection of: qualitative information; and contextual information to monitor equity • Assure the monitoring of changes over time and progress of particular student cohorts
  18. 18. 3. OECD School Resources Review
  19. 19. OECD School Resources Review: Objectives Overarching policy question: “What policies best ensure that school resources are effectively used to improve student outcomes?” • Key issues for analysis o Funding of school education [including budgeting in education] Level of resources; sources of revenue; education budgeting procedures; forecasting resource needs; resource strategy; distribution of funding across administrative levels, education levels, sectors and individual schools; monitoring of resource use; capacity for resource management; transparency and reporting. o Management of human resources (e.g. teachers, school directors) o Organisation of school network (location, size of schools) The Review will: • Synthesise research-based evidence on effective resource use in the school sector and disseminate this knowledge among countries • Identify innovative and successful policy initiatives and practices • Identify policy options for policy makers to consider.
  20. 20. Context Economic Demographic Political Cultural Governance OECD School Resources Review: Analytical framework and Key Issues Optimal outputs Access, participation, completion Learning outcomes Labour market & social outcomes Resource Distribution Resource Utilisation Resource Management Education System Goals
  21. 21. OECD School Resources Review: Participation • Austria • Belgium (Flemish Community) • Chile • Czech Republic • Denmark • Estonia • Kazakhstan • Lithuania • Slovak Republic • Uruguay Country Reviews • Belgium (French Community) • Iceland • Luxembourg • Spain • Sweden Country Background Reports
  22. 22. 4. Funding of School Education
  23. 23. Funding of School Education A diversity of channels for funding distribution: Central government to line ministries From Ministry of Education to local authorities To schools through funding formulas A range of factors used in Funding Formulae, including: • Number of students, class size, characteristics of student population, qualifications of teachers • Type of educational offer (e.g. VET vs General programmes), school location, size of school Performance measures typically not used Requires high-quality and reliable data at the school level
  24. 24. • Comparative indicators of school performance can miss important aspects of school quality – Standardised performance measures • Often limited to discrete areas of student learning objectives • Often cross-sectional and do not allow monitoring of school progress • Often do not identify impact of the school on student outcomes (value-added measures are needed) • Use of performance measures for accountability: • Not strongly linked to financial rewards/sanctions • Very different policies on how these feed into decisions on possible school closure • Closer supervision of schools with quality concerns • Reward of more autonomy to schools with good evaluation results • Care with accountability uses of school performance measures • Over emphasis on what is assessed in performance measures; narrowing of the curriculum; teaching to the test; hinder innovation. • Stigmatisation of particular schools and unintended impacts on parental choice of schools. Complacency of schools performing well on such measures. School Evaluation: Use of performance measures
  25. 25. Thank you for your attention!