The high cost of low educational performanceLondon, 9 February 2010Andreas SchleicherEducation Policy Advisor of the OECD Secretary-General
In the current economic environment……	Opportunity costs for education decline Dominated by lost earnings…	Labour-market entry becomes more difficultas young graduates compete with experienced workers…	Job prospects for less qualified deteriorate…	Young people with lower qualifications who become unemployed are likely to spend long time out of workIn most countries over half of low-qualified unemployed 25-34-year-olds are long-term unemployed …	Higher risks for systems with significant work-based training…	Gaps in educational attainment between younger and older cohorts likely to widen .
Know why you are lookingThe yardstick for success is no longer just improvement by national standards……	but the best performing education systems globallyKnow what you are looking forThe kind of ‘human capital’ that makes a difference for individuals and nationsHow do we know that we found it?Gauging impactImplicationsUnderstanding what contributes to the success of education systems and improving performance .
Know why you are lookingThe yardstick for success is no longer just improvement by national standards……	but the best performing education systems globally
A world of change – highereducationExpenditure per student at tertiary level (USD)Cost per studentGraduate supplyTertiary-type A graduation rate
A world of change – highereducationExpenditure per student at tertiary level (USD)United StatesCost per studentFinlandGraduate supplyTertiary-type A graduation rate
A world of change – highereducationExpenditure per student at tertiary level (USD)AustraliaFinlandUnited KingdomTertiary-type A graduation rate
A world of change – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
A world of change – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
A world of change – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
A world of change – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
A world of change – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
A world of change – highereducationExpenditure per student at tertiary level (USD)United StatesAustraliaUnited KingdomFinlandTertiary-type A graduation rate
Moving targetsFuture supply of college graduates
Components of the private net present value for a male with higher education27K$56K$170K$105K$35K$26K$367K$Net present value in USD equivalent
Public cost and benefits for a male obtaining post-secondary educationPublic costsPublic benefitsNet present value, USD equivalent(numbers in orange shownegative values)USD equivalent
Know what you are looking forThe kind of human capital that makes a difference for people and nations
Latin America then…Hanushek 2009
Latin America then and now…Hanushek 2009
Latin America then and now…Why quality is the keyHanushek 2009
How the demand for skills has changedEconomy-wide measures of routine and non-routine task input (US)Mean task input as percentiles of  the 1960 task distributionThe dilemma of schools:The skills that are easiest to teach and test are also the ones that are easiest to digitise, automate and outsource (Levy and Murnane)
OECD’s PISA assessment of the knowledge and skills of 15-year-oldsCoverage of world economy83%77%81%85%86%87%
High science performanceAverage performanceof 15-year-olds in science – extrapolate and apply… 18 countries perform below this lineLow science performance
High science performanceAverage performanceof 15-year-olds in science – extrapolate and applyHigh average performanceLarge socio-economic disparitiesHigh average performanceHigh social equityStrong socio-economic impact on student performanceSocially equitable distribution of learning opportunitiesLow average performanceLarge socio-economic disparitiesLow average performanceHigh social equityLow science performance
High science performanceDurchschnittliche Schülerleistungen im Bereich MathematikHigh average performanceLarge socio-economic disparitiesHigh average performanceHigh social equityStrong socio-economic impact on student performanceSocially equitable distribution of learning opportunitiesLow average performanceLarge socio-economic disparitiesLow average performanceHigh social equityLow science performance
Student performancePISA Index of socio-economic backgroundAdvantageDisadvantageSchool performance and socio-economic background GermanyStudent performance and students’ socio-economic background withinschoolsSchool performance and schools’ socio-economic backgroundStudent performance and students’ socio-economic backgroundSchools proportional to size
Student performancePISA Index of socio-economic backgroundAdvantageDisadvantageSchool performance and socio-economic background United KingdomStudent performance and students’ socio-economic background within schoolsSchool performance and schools’ socio-economic backgroundStudent performance and students’ socio-economic backgroundSchools proportional to size
Student performancePISA Index of socio-economic backgroundAdvantageDisadvantageSchool performance and socio-economic background FinlandStudent performance and students’ socio-economic background within schoolsSchool performance and schools’ socio-economic backgroundStudent performance and students’ socio-economic backgroundSchools proportional to size
OECD Level 2OECD Level 6IdentifyingRecognising issues that can be investigated scientificallyIdentifying keywords in a scientific investigationRecognising the key features of a scientific investigationExplainingApplying knowledge of science in a situationDescribing or interpreting phenomena scientifically or predicting changeUsing evidenceInterpreting scientific evidence and drawing conclusionsIdentifying the assumptions, evidence and reasoning behind conclusionsStudents can determine ifscientific measurement can be applied to a given variable in an investigation. Students can appreciate the relationship between a simple model and the phenomenon it is modelling. Students can demonstrate ability to understand and articulate the complex modelling inherent in the design of an investigation.Context- Personal Social/public
 Global   CompetenciesIdentify scientific issues
Explain phenomena scientifically
Use scientific evidenceStudents can recall anappropriate, tangible, scientific fact applicable in a simple and straightforward context and can use it to explain or predict an outcome.Students can draw ona range of abstract scientific knowledge and concepts andthe relationships between these in developing explanations ofprocessesKnowledgeKnowledge of science
Knowledge about scienceAttitudes-Interest in science-Support for scientific enquiry-ResponsibilityStudents demonstrateability to compare and  differentiate among competing explanations byexamining supporting evidence. They can formulate arguments by synthesising evidence from multiplesources.Students can point to an obvious feature in a simple table in support of a given statement. They are able to recognise if a set of given characteristics apply to the function of everydayartifacts.
Top and bottom performers in scienceThese students can consistently identify, explain and apply scientific knowledge, link different information sources and explanations and use evidence from these to justify decisions, demonstrate advanced scientific thinking in unfamiliar situations…These students often confuse key features of a scientific investigation, apply incorrect information, mix personal beliefs with facts in support of a position… Large prop. of poor perf.Large proportion of top performers20
How do we know that we found it?To what extent knowledge and skills matter for the success of individuals and economies
Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15 (Canada)after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group PISA Level 1)Odds ratioCollege entrySchool marks at age 15PISA performance at age 15
Modelling the impactProgrammes to improve cognitive skills through schools take time to implement and to have their impact on students.Assume that it will take 20 years to implement reformThe impact of improved skills will not be realised until the students with greater skills move into the labour forceAssume that improved PISA performance will result in improved skill-based of 2.5% of the labour-force each yearThe economy will respond over time as new technologies are developed and implemented, making use of the new higher skillsEstimate the total gains over the lifetime of the generation born this year .
Relationship between test performance and economic outcomesAnnual improved GDP from raising performance by 25 PISA pointsPercent addition to GDP
Increase average performance by 25 PISA points (Total 115 trillion $)bn$
Catching up with Finland (total 260 trillion $)bn$
Catching up with Finland(in percent of GDP)% currrent GDP

The High Cost of Low Educational Performance

  • 1.
    The high costof low educational performanceLondon, 9 February 2010Andreas SchleicherEducation Policy Advisor of the OECD Secretary-General
  • 2.
    In the currenteconomic environment…… Opportunity costs for education decline Dominated by lost earnings… Labour-market entry becomes more difficultas young graduates compete with experienced workers… Job prospects for less qualified deteriorate… Young people with lower qualifications who become unemployed are likely to spend long time out of workIn most countries over half of low-qualified unemployed 25-34-year-olds are long-term unemployed … Higher risks for systems with significant work-based training… Gaps in educational attainment between younger and older cohorts likely to widen .
  • 3.
    Know why youare lookingThe yardstick for success is no longer just improvement by national standards…… but the best performing education systems globallyKnow what you are looking forThe kind of ‘human capital’ that makes a difference for individuals and nationsHow do we know that we found it?Gauging impactImplicationsUnderstanding what contributes to the success of education systems and improving performance .
  • 4.
    Know why youare lookingThe yardstick for success is no longer just improvement by national standards…… but the best performing education systems globally
  • 5.
    A world ofchange – highereducationExpenditure per student at tertiary level (USD)Cost per studentGraduate supplyTertiary-type A graduation rate
  • 6.
    A world ofchange – highereducationExpenditure per student at tertiary level (USD)United StatesCost per studentFinlandGraduate supplyTertiary-type A graduation rate
  • 7.
    A world ofchange – highereducationExpenditure per student at tertiary level (USD)AustraliaFinlandUnited KingdomTertiary-type A graduation rate
  • 8.
    A world ofchange – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
  • 9.
    A world ofchange – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
  • 10.
    A world ofchange – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
  • 11.
    A world ofchange – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
  • 12.
    A world ofchange – highereducationExpenditure per student at tertiary level (USD)Tertiary-type A graduation rate
  • 13.
    A world ofchange – highereducationExpenditure per student at tertiary level (USD)United StatesAustraliaUnited KingdomFinlandTertiary-type A graduation rate
  • 14.
    Moving targetsFuture supplyof college graduates
  • 15.
    Components of theprivate net present value for a male with higher education27K$56K$170K$105K$35K$26K$367K$Net present value in USD equivalent
  • 16.
    Public cost andbenefits for a male obtaining post-secondary educationPublic costsPublic benefitsNet present value, USD equivalent(numbers in orange shownegative values)USD equivalent
  • 18.
    Know what youare looking forThe kind of human capital that makes a difference for people and nations
  • 19.
  • 20.
    Latin America thenand now…Hanushek 2009
  • 21.
    Latin America thenand now…Why quality is the keyHanushek 2009
  • 22.
    How the demandfor skills has changedEconomy-wide measures of routine and non-routine task input (US)Mean task input as percentiles of the 1960 task distributionThe dilemma of schools:The skills that are easiest to teach and test are also the ones that are easiest to digitise, automate and outsource (Levy and Murnane)
  • 23.
    OECD’s PISA assessmentof the knowledge and skills of 15-year-oldsCoverage of world economy83%77%81%85%86%87%
  • 24.
    High science performanceAverageperformanceof 15-year-olds in science – extrapolate and apply… 18 countries perform below this lineLow science performance
  • 25.
    High science performanceAverageperformanceof 15-year-olds in science – extrapolate and applyHigh average performanceLarge socio-economic disparitiesHigh average performanceHigh social equityStrong socio-economic impact on student performanceSocially equitable distribution of learning opportunitiesLow average performanceLarge socio-economic disparitiesLow average performanceHigh social equityLow science performance
  • 26.
    High science performanceDurchschnittlicheSchülerleistungen im Bereich MathematikHigh average performanceLarge socio-economic disparitiesHigh average performanceHigh social equityStrong socio-economic impact on student performanceSocially equitable distribution of learning opportunitiesLow average performanceLarge socio-economic disparitiesLow average performanceHigh social equityLow science performance
  • 27.
    Student performancePISA Indexof socio-economic backgroundAdvantageDisadvantageSchool performance and socio-economic background GermanyStudent performance and students’ socio-economic background withinschoolsSchool performance and schools’ socio-economic backgroundStudent performance and students’ socio-economic backgroundSchools proportional to size
  • 28.
    Student performancePISA Indexof socio-economic backgroundAdvantageDisadvantageSchool performance and socio-economic background United KingdomStudent performance and students’ socio-economic background within schoolsSchool performance and schools’ socio-economic backgroundStudent performance and students’ socio-economic backgroundSchools proportional to size
  • 29.
    Student performancePISA Indexof socio-economic backgroundAdvantageDisadvantageSchool performance and socio-economic background FinlandStudent performance and students’ socio-economic background within schoolsSchool performance and schools’ socio-economic backgroundStudent performance and students’ socio-economic backgroundSchools proportional to size
  • 30.
    OECD Level 2OECDLevel 6IdentifyingRecognising issues that can be investigated scientificallyIdentifying keywords in a scientific investigationRecognising the key features of a scientific investigationExplainingApplying knowledge of science in a situationDescribing or interpreting phenomena scientifically or predicting changeUsing evidenceInterpreting scientific evidence and drawing conclusionsIdentifying the assumptions, evidence and reasoning behind conclusionsStudents can determine ifscientific measurement can be applied to a given variable in an investigation. Students can appreciate the relationship between a simple model and the phenomenon it is modelling. Students can demonstrate ability to understand and articulate the complex modelling inherent in the design of an investigation.Context- Personal Social/public
  • 31.
    Global CompetenciesIdentify scientific issues
  • 32.
  • 33.
    Use scientific evidenceStudentscan recall anappropriate, tangible, scientific fact applicable in a simple and straightforward context and can use it to explain or predict an outcome.Students can draw ona range of abstract scientific knowledge and concepts andthe relationships between these in developing explanations ofprocessesKnowledgeKnowledge of science
  • 34.
    Knowledge about scienceAttitudes-Interestin science-Support for scientific enquiry-ResponsibilityStudents demonstrateability to compare and differentiate among competing explanations byexamining supporting evidence. They can formulate arguments by synthesising evidence from multiplesources.Students can point to an obvious feature in a simple table in support of a given statement. They are able to recognise if a set of given characteristics apply to the function of everydayartifacts.
  • 35.
    Top and bottomperformers in scienceThese students can consistently identify, explain and apply scientific knowledge, link different information sources and explanations and use evidence from these to justify decisions, demonstrate advanced scientific thinking in unfamiliar situations…These students often confuse key features of a scientific investigation, apply incorrect information, mix personal beliefs with facts in support of a position… Large prop. of poor perf.Large proportion of top performers20
  • 37.
    How do weknow that we found it?To what extent knowledge and skills matter for the success of individuals and economies
  • 38.
    Increased likelihood ofpostsec. particip. at age 19/21 associated with PISA reading proficiency at age 15 (Canada)after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group PISA Level 1)Odds ratioCollege entrySchool marks at age 15PISA performance at age 15
  • 39.
    Modelling the impactProgrammesto improve cognitive skills through schools take time to implement and to have their impact on students.Assume that it will take 20 years to implement reformThe impact of improved skills will not be realised until the students with greater skills move into the labour forceAssume that improved PISA performance will result in improved skill-based of 2.5% of the labour-force each yearThe economy will respond over time as new technologies are developed and implemented, making use of the new higher skillsEstimate the total gains over the lifetime of the generation born this year .
  • 40.
    Relationship between testperformance and economic outcomesAnnual improved GDP from raising performance by 25 PISA pointsPercent addition to GDP
  • 41.
    Increase average performanceby 25 PISA points (Total 115 trillion $)bn$
  • 42.
    Catching up withFinland (total 260 trillion $)bn$
  • 43.
    Catching up withFinland(in percent of GDP)% currrent GDP
  • 44.
    Raise everyone tominimum of 400 PISA pointsbn$
  • 45.
    Raise everyone tominimum of 400 PISA points% currrent GDP
  • 46.
    Some conclusionsThe highereconomic outcomes that improved student performance entails dwarf the dimensions of economic cyclesEven if the estimated impacts of skills were twice as large as the true underlying causal impact on growth, the resulting present value of successful school reform still far exceeds any conceivable costs of improvement.
  • 48.
    ImplicationsUnderstanding what contributesto the success of education systems and improving performance
  • 49.
    Money matters -but other things do tooQuestion:If better education results in more money, Does more money result in better education?
  • 50.
    Spending choices onsecondary schoolsContribution of various factors to upper secondary teacher compensation costsper student as a percentage of GDP per capita (2004)Percentage points
  • 51.
    High ambitions anduniversal standardsRigor, focus and coherenceGreat systems attract great teachers and provide access to best practice and quality professional development
  • 52.
    Challenge and supportStrongsupportPoor performanceImprovements idiosyncraticStrong performanceSystemic improvementLowchallengeHighchallengePoor performanceStagnationConflictDemoralisationWeak support
  • 53.
    International Best PracticeThepastPrincipals who are trained, empowered, accountable and provide instructional leadership
  • 54.
    Principals who manage‘a building’, who have little training and preparation and are accountable but not empowered
  • 55.
    Attracting, recruiting andproviding excellent training for prospective teachers from the top third of the graduate distribution
  • 56.
    Attracting and recruitingteachers from the bottom third of the graduate distribution and offering training which does not relate to real classrooms
  • 57.
    Incentives, rules andfunding encourage a fair distribution of teaching talent
  • 58.
    The best teachersare in the most advantaged communitiesHuman capital
  • 59.
    International Best PracticeThepastExpectations of teachers are clear; consistent quality, strong professional ethic and excellent professional development focused on classroom practice
  • 60.
    Seniority and tenurematter more than performance; patchy professional development; wide variation in quality
  • 61.
    Teachers and thesystem expect every child to succeed and intervene preventatively to ensure this
  • 62.
    Wide achievement gaps,just beginning to narrow but systemic and professional barriers to transformation remain in placeHuman capital (cont…)
  • 63.
    High ambitionsDevolved responsibility,theschool as the centre of actionAccountability and intervention in inverse proportion to successAccess to best practice and quality professional development
  • 64.
    School autonomy, standards-basedexaminations and science performanceSchool autonomy in selecting teachers for hirePISA score in science
  • 65.
    Public and privateschools%Score point differencePublic schools perform betterPrivate schools perform better
  • 66.
    Pooled international dataset,effects of selected school/system factors on science performance after accounting for all other factors in the modelSchool principal’s positive evaluation of quality of educational materials(gross only)Schools with more competing schools(gross only)Schools with greater autonomy (resources)(gross and net)School activities to promote science learning(gross and net)One additional hour of self-study or homework (gross and net)One additional hour of science learning at school (gross and net)School results posted publicly (gross and net)Academically selective schools (gross and net) but no system-wide effectSchools practicing ability grouping (gross and net)One additional hour of out-of-school lessons (gross and net)20Each additional 10% of public funding(gross only)School principal’s perception that lack of qualified teachers hinders instruction(gross only)Effect after accounting for the socio-economic background of students, schools and countriesMeasured effectOECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a
  • 67.
    Strong ambitionsDevolvedresponsibility,the schoolas the centre of actionIntegrated educational opportunities From prescribed forms of teaching and assessment towards personalised learningAccountabilityAccess to best practice and quality professional development
  • 68.
    High science performanceDurchschnittlicheSchülerleistungen im Bereich MathematikHigh average performanceLarge socio-economic disparitiesHigh average performanceHigh social equityStrong socio-economic impact on student performanceSocially equitable distribution of learning opportunitiesEarly selection and institutional differentiation High degree of stratification Low degree of stratificationLow average performanceLarge socio-economic disparitiesLow average performanceHigh social equityLow science performance
  • 69.
  • 70.
    www.oecd.org; www.pisa.oecd.orgAllnational and international publicationsThe complete micro-level databaseemail: pisa@oecd.orgAndreas.Schleicher@OECD.org… and remember: Without data, you are just another person with an opinionThank you !

Editor's Notes

  • #4 I want to reflect briefly on what motivates us to look at education in a global context. In the flat world where all work that can be digitized, automated and outsourced can now be done by the most effective and competitive individuals or enterprises, wherever on the globe they are located, the yardstick for educational success is no longer just improvement by national educational standards, but the best performing education systems internationally.But the challenge is not just to produce more of the same degrees and qualifications, it is also about delivering new types of human capital that can put firms and countries on a path to sustainable, smarter and greener growth. A generation ago, teachers could expect that what they taught would last for a lifetime of their students. Today, they need to prepare students for more rapid change than ever before, for jobs that have not yet been created, to use technologies that have not yet been invented, and to solve problems that we don’t yet know will arise. Of course, state-of-the-art knowledge will always remain important. But education today needs to be also about ways of thinking, involving creativity, critical thinking, problem-solving and decision-making; about ways of working, including communication and collaboration; about tools for working, including the capacity to recognise and exploit the potential of new technologies; and, last but not least, about the capacity to live in a multi-faceted world as active and responsible citizens.But how do we know that we found what we are looking for? Everybody is claiming to know what kind of education is most productive, but how exactly can we prove that.Last but not least, what consequences do we draw from this for school systems?
  • #5 Let me begin by showing you how the global talent pool has changed, in recent years
  • #6 The pace of change is most clearly visible in higher education, and I want to bring two more dimensions into the picture here. Each dot on this chart represents one country. The horizontal axis shows you the college graduation rate, the proportion of an age group that comes out of the system with a college degree. The vertical axis shows you how much it costs to educate a graduate per year.
  • #7 *Lets now add where the money comes from into the picture, the larger the dot, the larger the share of private spending on college education, such as tuition.The chart shows the US as the country with the highest college graduation rate, and the highest level of spending per student. The US is also among the countries with the largest share of resources generated through the private sector. That allows the US to spend roughly twice as much per student as Europe. US, FinlandThe only thing I have not highlighted so far is that this was the situation in 1995. And now watch this closely as you see how this changed between 1995 and 2005.
  • #8 You see that in 2000, five years, later, the picture looked very different. While in 1995 the US was well ahead of any other country – you see that marked by the dotted circle, in 2000 several other countries had reached out to this frontier. Look at Australia, in pink.
  • #13 Thatwasallveryquick, letusgothroughthisdevelopmentonceagain
  • #14 Thatwasallveryquick, letusgothroughthisdevelopmentonceagain
  • #15 This is where China, the European Union, India and the US stood in terms of the number of high school graduates in 2003. This is how the picture is likely to look in 2010, and this is what we project for 2015.
  • #17 Let me then address the issue of what we know about the kind of human capital that will make a difference.
  • #18 Let us go back to the 1960s. The chart shows you the wealth of world regions and the average years of schooling in these regions, which is the most traditional measure of human capital. Have a look at Latin America, it ranked third in wealth and third in years of schooling, so in the 1960s the world seemed pretty much in order.
  • #19 But when you look at economic growth between 1960 and 2000, you see that something went wrong. Despite the fact that Latin America did well in terms of years of schooling, only Sub-Saharan Africa did worse in terms of economic growth. So in 2000, Latin America had fallen back considerably in terms of GDP per capita.You can draw two conclusions from this: Either education is not as important for economic growth as we thought, or we have for a long time been measuring the wrong thing.
  • #20 Now let me add one additional element, and that is a measure of the quality of education, in the form of the score of the different world regions on international tests like PISA or TIMSS. And you see now that the world looks in order again, there seems a close relationship between test scores and economic growth. You can see that even more clearly when you put this into graphical form. This is one of the charts produced by Professor Hanushek. And, as Professor Hanushek will explain, the relationship holds even when you account for other factors, it even holds when you compare growth in economies with growth in learning outcomes, which is the closest we can come to examining causality.So what this tells you is that it is not simply years of schooling or the number of graduates we produce, but indeed the quality of learning outcomes that counts.
  • #21 Levy and Murnane show how the composition of the US work force has changed. What they show is that, between 1970 and 2000, work involving routine manual input, the jobs of the typical factory worker, was down significantly. Non-routine manual work, things we do with our hands, but in ways that are not so easily put into formal algorithms, was down too, albeit with much less change over recent years – and that is easy to understand because you cannot easily computerise the bus driver or outsource your hairdresser. All that is not surprising, but here is where the interesting story begins: Among the skill categories represented here, routine cognitive input, that is cognitive work that you can easily put into the form of algorithms and scripts saw the sharpest decline in demand over the last couple of decades, with a decline by almost 8% in the share of jobs. So those middle class white collar jobs that involve the application of routine knowledge, are most at threat today. And that is where schools still put a lot of their focus and what we value in multiple choice accountability systems.The point here is, that the skills that are easiest to teach and test are also the skills that are easiest to digitise, automatise and offshore. If that is all what we do in school, we are putting our youngsters right up for competition with computers, because those are the things computers can do better than humans, and our kids are going to loose out before they even started. Where are the winners in this process? These are those who engage in expert thinking – the new literacy of the 21st century, up 8% - and complex communication, up almost 14%.
  • #22 At the OECD, we are measuring skills, with a focus on those non-routing cognitive skills, regularly through our PISA programme, now the most comprehensive international assessment of the quality of education. Every three years, we test roughly half a million of children in OECD countries in key competencies, and that’s not simply about checking whether students have learned what they were recently taught, but we examine to what extent students can extrapolate from what they have learned and apply their knowledge and skills in novel settings. Here you see the countries which we can compare, and how the set of countries being compared has expanded.
  • #33 How do we know that we know?I want to distinguish here between the impact knowledge and skills such as those assessed by PISA have for the success of individuals, on the one hand, and economies, on the other.
  • #34 The best way to find out whether what students have learned at school matters for their life is to actuallywatch what happens to them after they leave school. This is exactly what we have done that with around 30,000 students in Canada. We tested them in the year 2000 when they were 15 years old in reading, math and science, and since then we are following up with them each year on what choices they make and how successful they are in their transition from school to higher education and work.The horizontal axis shows you the PISA level which 15-year-old Canadians had scored in 2000. Level 2 is the baseline level on the PISA reading test and Level 5 the top level in reading.The red bar shows you how many times more successful someone who scored Level 2 at age 15 was at age 19 to have made a successful transition to university, as compared to someone who did not make it to the baseline PISA level 1. And to ensure that what you see here is not simply a reflection of social background, gender, immigration or school engagement, we have already statistically accounted for all of these factors. The orange bar. …How would you expect the picture to be like at age 21? We are talking about test scores here, but for a moment, lets go back to the judgements schools make on young people, for example through school marks. You can do the same thing here, you can see how well school marks at age 15 predict the subsequent success of youths. You see that there is some relationship as well, but that it is much less pronounced than when we use the direct measure of skills.