1. Seeing your education system in the mirror of other systemsExamples from the OECD Jakarta, May 11 2010 Andreas SchleicherEducation Policy Advisor of the OECD Secretary-General
2. Domain 1 Individual learner LevelA LevelB Instructional settings LevelC Schools, other institutions Country or system LevelD Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour Socio-economic background of learners Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate The learning environment at school Community and school characteristics Output and performance of institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies
3. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD
4. A world of change – highereducation Expenditure per student at tertiary level (USD) Cost per student Graduate supply Tertiary-type A graduation rate
5. A world of change – highereducation Expenditure per student at tertiary level (USD) United States Cost per student Finland Japan Graduate supply Tertiary-type A graduation rate
6. A world of change – highereducation Expenditure per student at tertiary level (USD) Australia Finland United Kingdom Tertiary-type A graduation rate
7. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate
8. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate
9. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate
10. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate
11. A world of change – highereducation Expenditure per student at tertiary level (USD) Tertiary-type A graduation rate
12. A world of change – highereducation What about international students? Expenditure per student at tertiary level (USD) United States Australia A A United Kingdom Finland A Tertiary-type A graduation rate
13. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD
14. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD
15. Components of the private net present value for a male with higher education 27K$ 56K$ 170K$ 105K$ 35K$ 26K$ 367K$ Net present value in USD equivalent
16. Public cost and benefits for a male obtaining post-secondary education Public costs Public benefits Net present value, USD equivalent (numbers in orange shownegative values) USD equivalent
17. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD
18. Spending choices on secondary schoolsContribution of various factors to upper secondary teacher compensation costsper student as a percentage of GDP per capita (2004) Percentage points
26. Relationships between teachers, heads, supervisors, inspectors and advisors etc.Infrastructure Teaching and learning Strategies Funding Definition of expected standards Competencies and content, relevance and links to “real world” Alignment of curricula with standards Quality of materials to support work of teachers Assessment approaches around the standards Assessment for learning Assessment of learning and monitoring Standards and intervention in inverse proportion to success Leadership and school ethos Devolution of first line responsibility for the quality of educational provision to the point of delivery Local leadership and teacher development Feedback systems which provide information about effectiveness and trends in outcomes Performance indicators, stakeholder surveys, professional evaluation Stakeholder involvement Human capital High quality learning for every student Teaching quality and professional development State and regional context Effective classroom practice Organisation of schooling Autonomy and accountability Curriculum and assessment
27. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD
28. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD
29. How the demand for skills has changedEconomy-wide measures of routine and non-routine task input (US) Mean task input as percentiles of the 1960 task distribution The dilemma of schools: The skills that are easiest to teach and test are also the ones that are easiest to digitise, automate and outsource (Levy and Murnane)
30. OECD’s PISA assessment of the knowledge and skills of 15-year-olds Coverage of world economy 83% 77% 81% 85% 86% 87%
31. Example Mathematics The real world The mathematical World Making the problem amenable to mathematical treatment A mathematical model A model of reality Understanding, structuring and simplifying the situation Using relevant mathematical tools to solve the problem A real situation Validating the results Mathematical results Real results Interpreting the mathematical results
32. High science performance Average performanceof 15-year-olds in science – extrapolate and apply Poland 2000 … 18 countries perform below this line Low science performance
33. High science performance Average performanceof 15-year-olds in science – extrapolate and apply High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low science performance
34. High science performance Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low science performance
35. Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics 20
36. Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics Variation of performance within schools Variation of performance between schools OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383.
41. Knowledge about scienceAttitudes -Interest in science -Support for scientific enquiry -Responsibility Students demonstrate ability to compare and differentiate among competing explanations by examining supporting evidence. They can formulate arguments by synthesising evidence from multiple sources. Students can point to an obvious feature in a simple table in support of a given statement. They are able to recognise if a set of given characteristics apply to the function of everyday artifacts.
42. Top and bottom performers in science These students can consistently identify, explain and apply scientific knowledge, link different information sources and explanations and use evidence from these to justify decisions, demonstrate advanced scientific thinking in unfamiliar situations… These students often confuse key features of a scientific investigation, apply incorrect information, mix personal beliefs with facts in support of a position… Large prop. of poor perf. Large proportion of top performers 20
43. Domain 1 Dimensions for educational benchmarking Domain 3 Domain 2 Antecedentscontextualise or constrain ed policy Policy Leversshape educational outcomes Outputs and Outcomesimpact of learning Quality and distribution of knowledge & skills Individ attitudes, engagement and behaviour LevelA Socio-economic background of learners Individual learner LevelB Student learning, teacher working conditions Quality of instructional delivery Teaching, learning practices and classroom climate Instructional settings The learning environment at school Community and school characteristics Output and performance of institutions LevelC Schools, other institutions National educ, social and economic context Social & economic outcomes of education Structures, resource alloc and policies Country or system LevelD
44. Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15 (Canada)after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group PISA Level 1) Odds ratioCollege entry School marks at age 15 PISA performance at age 15
45. Relationship between test performance and economic outcomesAnnual improved GDP from raising performance by 25 PISA points Percent addition to GDP
47. Public and private schools % Score point difference Public schools perform better Private schools perform better
48. School autonomy, standards-based examinations and science performanceSchool autonomy in selecting teachers for hire PISA score in science
49. Pooled international dataset, effects of selected school/system factors on science performance after accounting for all other factors in the model School principal’s positive evaluation of quality of educational materials(gross only) Schools with more competing schools(gross only) Schools with greater autonomy (resources)(gross and net) School activities to promote science learning(gross and net) One additional hour of self-study or homework (gross and net) One additional hour of science learning at school (gross and net) School results posted publicly (gross and net) Academically selective schools (gross and net) but no system-wide effect Schools practicing ability grouping (gross and net) One additional hour of out-of-school lessons (gross and net) 20 Each additional 10% of public funding(gross only) School principal’s perception that lack of qualified teachers hinders instruction(gross only) Effect after accounting for the socio-economic background of students, schools and countries Measured effect OECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a
51. High policy value A real-time assessment environment that bridges the gap between formative and summative assessment . Quick wins Must haves Examine individual, institutional and systemic factors associated with performance Extending the range of competencies through which quality is assessed Monitor educational progress Measuring growth in learning Low feasibility High feasibility Establish the relative standing of students and schools Assuming that every new skill domain is orthogonal to all others Money pits Low-hanging fruits Low policy value
52. High ambitions and universal standards Rigor, focus and coherence Great systems attract great teachers and provide access to best practice and quality professional development
53. Challenge and support Strong support Poor performance Improvements idiosyncratic Strong performance Systemic improvement Lowchallenge Highchallenge Poor performance Stagnation Conflict Demoralisation Weak support
54.
55. Principals who manage ‘a building’, who have little training and preparation and are accountable but not empowered
56. Attracting, recruiting and providing excellent training for prospective teachers from the top third of the graduate distribution
57. Attracting and recruiting teachers from the bottom third of the graduate distribution and offering training which does not relate to real classrooms
59. The best teachers are in the most advantaged communitiesHuman capital
60.
61. Seniority and tenure matter more than performance; patchy professional development; wide variation in quality
62. Teachers and the system expect every child to succeed and intervene preventatively to ensure this
63. Wide achievement gaps, just beginning to narrow but systemic and professional barriers to transformation remain in placeHuman capital (cont…)
64. Relatively few teachers participate in the kinds of professional development which they find has the largest impact on their work Comparison of teachers participating in professional development activities and teachers reporting moderate or high level impact by types of activity Figure 3.15
65. High ambitions Devolved responsibility,the school as the centre of action Accountability and intervention in inverse proportion to success Access to best practice and quality professional development
66. Local responsibility and national prescription Towards system-wide sustainable reform National prescription Schools today The industrial model, detailed prescription of what schools do Schools tomorrow? Building capacity Finland today Every school an effective school Schools leading reform
67. Strong ambitions Devolvedresponsibility,the school as the centre of action Integrated educational opportunities From prescribed forms of teaching and assessment towards personalised learning Accountability Access to best practice and quality professional development
68. High science performance Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Early selection and institutional differentiation High degree of stratification Low degree of stratification Low average performance Large socio-economic disparities Low average performance High social equity Low science performance
69. www.oecd.org; www.pisa.oecd.org All national and international publications The complete micro-level database email: pisa@oecd.org Andreas.Schleicher@OECD.org … and remember: Without data, you are just another person with an opinion Thank you !
Editor's Notes
The pace of change is most clearly visible in higher education, and I want to bring two more dimensions into the picture here. Each dot on this chart represents one country. The horizontal axis shows you the college graduation rate, the proportion of an age group that comes out of the system with a college degree. The vertical axis shows you how much it costs to educate a graduate per year.
*Lets now add where the money comes from into the picture, the larger the dot, the larger the share of private spending on college education, such as tuition.The chart shows the US as the country with the highest college graduation rate, and the highest level of spending per student. The US is also among the countries with the largest share of resources generated through the private sector. That allows the US to spend roughly twice as much per student as Europe. US, FinlandThe only thing I have not highlighted so far is that this was the situation in 1995. And now watch this closely as you see how this changed between 1995 and 2005.
You see that in 2000, five years, later, the picture looked very different. While in 1995 the US was well ahead of any other country – you see that marked by the dotted circle, in 2000 several other countries had reached out to this frontier. Look at Australia, in pink.
The best way to find out whether what students have learned at school matters for their life is to actuallywatch what happens to them after they leave school. This is exactly what we have done that with around 30,000 students in Canada. We tested them in the year 2000 when they were 15 years old in reading, math and science, and since then we are following up with them each year on what choices they make and how successful they are in their transition from school to higher education and work.The horizontal axis shows you the PISA level which 15-year-old Canadians had scored in 2000. Level 2 is the baseline level on the PISA reading test and Level 5 the top level in reading.The red bar shows you how many times more successful someone who scored Level 2 at age 15 was at age 19 to have made a successful transition to university, as compared to someone who did not make it to the baseline PISA level 1. And to ensure that what you see here is not simply a reflection of social background, gender, immigration or school engagement, we have already statistically accounted for all of these factors. The orange bar. …How would you expect the picture to be like at age 21? We are talking about test scores here, but for a moment, lets go back to the judgements schools make on young people, for example through school marks. You can do the same thing here, you can see how well school marks at age 15 predict the subsequent success of youths. You see that there is some relationship as well, but that it is much less pronounced than when we use the direct measure of skills.
This chart shows you the proportion of teachers who participated in various types of professional development over the last 18 months, with the bars showing the average across countries and the red dot showing the Mexican figures. So you see that just over 60% of Mexican teachers have engaged in some form of individual and collaborative research, just over 30% in qualification programmes, almost every teacher in informal dialogue to improve teaching, 70% in reading professional literature, and so on.These are impressive numbers. But do governments offer, and do teachers take up the kind of professional development that is actually most effective? The yellow bar shows you the proportion of teachers who think that the various types of professional development have a moderate to large impact on their development as a teacher. So you see that, while individual and collaborative research seems to have the largest impact (the yellow bar is long), participation rates here, shown by the blue bar, are comparatively low. The same is true for sustained qualification programs, these seem to make a genuine impact but few teachers pursue such courses. In contrast, lots of teachers participate in one-off seminars and workshops which much fewer teachers perceive to be of value.TALIS thus shows that we need to do better in matching the costs and benefit as well as supply and demand for professional development. Courses and workshopsProfessional development networkMentoring and peer observationObservation visits to other schoolsEducation conferences and seminars
But the balance between national prescription and schools leading reform is not an all-or-nothing. In fact, most school systems have started out with highly prescriptive education systems. But gradually the have moved towards building capacity and enabling schools to assume greater responsibility.