Your SlideShare is downloading. ×
2010 e-madrid (spanish presidency) - rev 1.0
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

2010 e-madrid (spanish presidency) - rev 1.0

585

Published on

Spanish EU Presidency discusses how indicators can help improve education

Spanish EU Presidency discusses how indicators can help improve education

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
585
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Such an OECD knowledge base needs to reflect three dimension, (1) an understanding of what countries are trying to achieve, (2) the evidence that we have, and (3) what we know about policy delivery in different contexts. As concerning the policy context: Do we understand what countries care about, what they intend to do about it and how they define success? Do we understand the policy context, and that is not just about the context as it stands, but how we anticipate its future development, because in an area like education where the delivery chain of reforms is so long, we need to think far into the future. Do we understand the strategic requirements for change, and do we judge various approaches to change technically feasible? And not everything that is feasible is politically suitable. And not everything that can be done is robust and cost-effective. With regard to the evidence: Are our indicators and analyses adequately capturing past and present performance of the education system vis a vis the reform goals, and have we been able to pin-point the drivers of performance and their underlying system activities? How does what we produce at the OECD, whether that is data, country reviews or thematic review relate to and add value to what other players provide?Finally, to what extent is our analysis useful and actionable in national contexts. Do we understand the challenges for reform delivery in countries and their delivery capacity? Is what we propose doable by real people in real situations and avoiding big time and energy traps?
  • The pace of change is most clearly visible in higher education, and I want to bring two more dimensions into the picture here. Each dot on this chart represents one country. The horizontal axis shows you the college graduation rate, the proportion of an age group that comes out of the system with a college degree. The vertical axis shows you how much it costs to educate a graduate per year.
  • *Lets now add where the money comes from into the picture, the larger the dot, the larger the share of private spending on college education, such as tuition.The chart shows the US as the country with the highest college graduation rate, and the highest level of spending per student. The US is also among the countries with the largest share of resources generated through the private sector. That allows the US to spend roughly twice as much per student as Europe. US, FinlandThe only thing I have not highlighted so far is that this was the situation in 1995. And now watch this closely as you see how this changed between 1995 and 2005.
  • You see that in 2000, five years, later, the picture looked very different. While in 1995 the US was well ahead of any other country – you see that marked by the dotted circle, in 2000 several other countries had reached out to this frontier. Look at Australia, in pink.
  • Thatwasallveryquick, letusgothroughthisdevelopmentonceagain
  • The best way to find out whether what students have learned at school matters for their life is to actuallywatch what happens to them after they leave school. This is exactly what we have done that with around 30,000 students in Canada. We tested them in the year 2000 when they were 15 years old in reading, math and science, and since then we are following up with them each year on what choices they make and how successful they are in their transition from school to higher education and work.The horizontal axis shows you the PISA level which 15-year-old Canadians had scored in 2000. Level 2 is the baseline level on the PISA reading test and Level 5 the top level in reading.The red bar shows you how many times more successful someone who scored Level 2 at age 15 was at age 19 to have made a successful transition to university, as compared to someone who did not make it to the baseline PISA level 1. And to ensure that what you see here is not simply a reflection of social background, gender, immigration or school engagement, we have already statistically accounted for all of these factors. The orange bar. …How would you expect the picture to be like at age 21? We are talking about test scores here, but for a moment, lets go back to the judgements schools make on young people, for example through school marks. You can do the same thing here, you can see how well school marks at age 15 predict the subsequent success of youths. You see that there is some relationship as well, but that it is much less pronounced than when we use the direct measure of skills.
  • Let us go back to the 1960s. The chart shows you the wealth of world regions and the average years of schooling in these regions, which is the most traditional measure of human capital. Have a look at Latin America, it ranked third in wealth and third in years of schooling, so in the 1960s the world seemed pretty much in order.
  • But when you look at economic growth between 1960 and 2000, you see that something went wrong. Despite the fact that Latin America did well in terms of years of schooling, only Sub-Saharan Africa did worse in terms of economic growth. So in 2000, Latin America had fallen back considerably in terms of GDP per capita.You can draw two conclusions from this: Either education is not as important for economic growth as we thought, or we have for a long time been measuring the wrong thing.
  • Now let me add one additional element, and that is a measure of the quality of education, in the form of the score of the different world regions on international tests like PISA or TIMSS. And you see now that the world looks in order again, there seems a close relationship between test scores and economic growth. You can see that even more clearly when you put this into graphical form. This is one of the charts produced by Professor Hanushek. And, as Professor Hanushek will explain, the relationship holds even when you account for other factors, it even holds when you compare growth in economies with growth in learning outcomes, which is the closest we can come to examining causality.So what this tells you is that it is not simply years of schooling or the number of graduates we produce, but indeed the quality of learning outcomes that counts.
  • Transcript

    • 1. Indicators of Education SystemsSeeing your education system in the mirror of other systems
      Madrid, 24 May 2010
      Andreas SchleicherEducation Policy Advisor of the OECD Secretary-General
    • 2. Indicators of education systems
      Or the Alchemists’ Stone?
      The Holy Grail?
      Know why you are looking
      You cannot improve what you cannot measure
      The yardstick for success is no longer just improvement by national standards but the best performing education systems globally
      Know what you are looking for
      Systemic framework
      Policy relevance
      Cross-country and cross-cultural validity
      Know howyou will recognise it when you find it
      What makes a good indicator system?
      Implications and lessons learned .
    • 3. Know what you are looking for
      The Holy Grail was a well-described object, and there was only one true grail…
    • 4. More than compilations of statistics
      • Do we understand the policy objectives
      • 5. What countries care about, what they intend to do about it and how they define success
      • 6. Do we understand the policy context and can we anticipate its future development ?
      • 7. Do we understand the strategic requirements for change?
      • 8. Are these
      • 9. Technically feasible?
      • 10. Politically and socially suitable?
      • 11. Robust and cost-effective?
      • 12. Do we understand the delivery challenge and delivery capacity?
      • 13. Nature and size of the barriers that systems face to deliver reform goals
      • 14. Can what works in one country by done in another by real people in real situations?
      • 15. Avoiding big time and energy traps?
      • 16. Do we understand past and present performance vis a vis the policy goals as well as the drivers of performance and their underlying system activities?
      • 17. What is added value of international comparisons?
    • Domain 1
      Individual learner
      LevelA
      LevelB
      Instructional settings
      LevelC
      Schools, other institutions
      Country or system
      LevelD
      Dimensions of an indicators framework
      Domain 3
      Domain 2
      Antecedentscontextualise or constrain ed policy
      Policy Leversshape educational outcomes
      Outputs and Outcomesimpact of learning
      Quality and distribution of knowledge & skills
      Individ attitudes, engagement and behaviour
      Socio-economic background of learners
      Student learning, teacher working conditions
      Quality of instructional delivery
      Teaching, learning practices and classroom climate
      The learning environment at school
      Community and school characteristics
      Output and performance of institutions
      National educ, social and economic context
      Social & economic outcomes of education
      Structures, resource alloc and policies
    • 18. Domain 1
      Dimensions of an indicators framework
      Domain 3
      Domain 2
      Antecedentscontextualise or constrain ed policy
      Policy Leversshape educational outcomes
      Outputs and Outcomesimpact of learning
      Quality and distribution of knowledge & skills
      Individ attitudes, engagement and behaviour
      LevelA
      Socio-economic background of learners
      Individual learner
      LevelB
      Student learning, teacher working conditions
      Quality of instructional delivery
      Teaching, learning practices and classroom climate
      Instructional settings
      The learning environment at school
      Community and school characteristics
      Output and performance of institutions
      LevelC
      Schools, other institutions
      National educ, social and economic context
      Social & economic outcomes of education
      Structures, resource alloc and policies
      Country or system
      LevelD
    • 19. A world of change – highereducation
      Expenditure per student at tertiary level (USD)
      Cost per student
      Graduate supply
      Tertiary-type A graduation rate
    • 20. A world of change – highereducation
      Expenditure per student at tertiary level (USD)
      United States
      Cost per student
      Finland
      Japan
      Spain
      Graduate supply
      Tertiary-type A graduation rate
    • 21. A world of change – highereducation
      Expenditure per student at tertiary level (USD)
      Australia
      Finland
      United Kingdom
      Tertiary-type A graduation rate
    • 22. A world of change – highereducation
      Expenditure per student at tertiary level (USD)
      Tertiary-type A graduation rate
    • 23. A world of change – highereducation
      Expenditure per student at tertiary level (USD)
      Tertiary-type A graduation rate
    • 24. A world of change – highereducation
      Expenditure per student at tertiary level (USD)
      Tertiary-type A graduation rate
    • 25. A world of change – highereducation
      Expenditure per student at tertiary level (USD)
      Tertiary-type A graduation rate
    • 26. A world of change – highereducation
      Expenditure per student at tertiary level (USD)
      Tertiary-type A graduation rate
    • 27. A world of change – highereducation
      What about international students?
      Expenditure per student at tertiary level (USD)
      United States
      Australia
      A
      A
      United Kingdom
      Finland
      Spain
      A
      Tertiary-type A graduation rate
    • 28. Domain 1
      Dimensions of an indicators framework
      Domain 3
      Domain 2
      Antecedentscontextualise or constrain ed policy
      Policy Leversshape educational outcomes
      Outputs and Outcomesimpact of learning
      Quality and distribution of knowledge & skills
      Individ attitudes, engagement and behaviour
      LevelA
      Socio-economic background of learners
      Individual learner
      LevelB
      Student learning, teacher working conditions
      Quality of instructional delivery
      Teaching, learning practices and classroom climate
      Instructional settings
      The learning environment at school
      Community and school characteristics
      Output and performance of institutions
      LevelC
      Schools, other institutions
      National educ, social and economic context
      Social & economic outcomes of education
      Structures, resource alloc and policies
      Country or system
      LevelD
    • 29. Domain 1
      Dimensions of an indicators framework
      Domain 3
      Domain 2
      Antecedentscontextualise or constrain ed policy
      Policy Leversshape educational outcomes
      Outputs and Outcomesimpact of learning
      Quality and distribution of knowledge & skills
      Individ attitudes, engagement and behaviour
      LevelA
      Socio-economic background of learners
      Individual learner
      LevelB
      Student learning, teacher working conditions
      Quality of instructional delivery
      Teaching, learning practices and classroom climate
      Instructional settings
      The learning environment at school
      Community and school characteristics
      Output and performance of institutions
      LevelC
      Schools, other institutions
      National educ, social and economic context
      Social & economic outcomes of education
      Structures, resource alloc and policies
      Country or system
      LevelD
    • 30. Components of the private net present value for a male with higher education
      27K$
      56K$
      170K$
      105K$
      35K$
      26K$
      367K$
      Net present value in USD equivalent
    • 31. Public cost and benefits for a male obtaining
      post-secondary education
      Public costs
      Public benefits
      Net present value, USD equivalent
      (numbers in orange shownegative values)
      USD equivalent
    • 32. Domain 1
      Dimensions of an indicators framework
      Domain 3
      Domain 2
      Antecedentscontextualise or constrain ed policy
      Policy Leversshape educational outcomes
      Outputs and Outcomesimpact of learning
      Quality and distribution of knowledge & skills
      Individ attitudes, engagement and behaviour
      LevelA
      Socio-economic background of learners
      Individual learner
      LevelB
      Student learning, teacher working conditions
      Quality of instructional delivery
      Teaching, learning practices and classroom climate
      Instructional settings
      The learning environment at school
      Community and school characteristics
      Output and performance of institutions
      LevelC
      Schools, other institutions
      National educ, social and economic context
      Social & economic outcomes of education
      Structures, resource alloc and policies
      Country or system
      LevelD
    • 33. Spending choices on secondary schoolsContribution of various factors to upper secondary teacher compensation costsper student as a percentage of GDP per capita (2004)
      Percentage points
    • 34. Domain 1
      Dimensions of an indicators framework
      Domain 3
      Domain 2
      Antecedentscontextualise or constrain ed policy
      Policy Leversshape educational outcomes
      Outputs and Outcomesimpact of learning
      Quality and distribution of knowledge & skills
      Individ attitudes, engagement and behaviour
      LevelA
      Socio-economic background of learners
      Individual learner
      LevelB
      Student learning, teacher working conditions
      Quality of instructional delivery
      Teaching, learning practices and classroom climate
      Instructional settings
      The learning environment at school
      Community and school characteristics
      Output and performance of institutions
      LevelC
      Schools, other institutions
      National educ, social and economic context
      Social & economic outcomes of education
      Structures, resource alloc and policies
      Country or system
      LevelD
    • 35. Domain 1
      Dimensions of an indicators framework
      Domain 3
      Domain 2
      Antecedentscontextualise or constrain ed policy
      Policy Leversshape educational outcomes
      Outputs and Outcomesimpact of learning
      Quality and distribution of knowledge & skills
      Individ attitudes, engagement and behaviour
      LevelA
      Socio-economic background of learners
      Individual learner
      LevelB
      Student learning, teacher working conditions
      Quality of instructional delivery
      Teaching, learning practices and classroom climate
      Instructional settings
      The learning environment at school
      Community and school characteristics
      Output and performance of institutions
      LevelC
      Schools, other institutions
      National educ, social and economic context
      Social & economic outcomes of education
      Structures, resource alloc and policies
      Country or system
      LevelD
    • 36. OECD’s PISA assessment of the knowledge and skills of 15-year-olds
      Coverage of world economy
      83%
      77%
      81%
      85%
      86%
      87%
    • 37. High science performance
      Average performanceof 15-year-olds in science – extrapolate and apply
      Poland 2000
      … 18 countries perform below this line
      Low science performance
    • 38. High science performance
      Average performanceof 15-year-olds in science – extrapolate and apply
      High average performance
      Large socio-economic disparities
      High average performance
      High social equity
      Strong socio-economic impact on student performance
      Socially equitable distribution of learning opportunities
      Low average performance
      Large socio-economic disparities
      Low average performance
      High social equity
      Low science performance
    • 39. High science performance
      Durchschnittliche Schülerleistungen im Bereich Mathematik
      High average performance
      Large socio-economic disparities
      High average performance
      High social equity
      Strong socio-economic impact on student performance
      Socially equitable distribution of learning opportunities
      Low average performance
      Large socio-economic disparities
      Low average performance
      High social equity
      Low science performance
    • 40. Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics
      20
    • 41. Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics
      Variation of performance within schools
      Variation of performance between schools
      OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383.
    • 42. Pooled international dataset, effects of selected school/system factors on science performance after accounting for all other factors in the model
      School principal’s positive evaluation of quality of educational materials(gross only)
      Schools with more competing schools(gross only)
      Schools with greater autonomy (resources)(gross and net)
      School activities to promote science learning(gross and net)
      One additional hour of self-study or homework (gross and net)
      One additional hour of science learning at school (gross and net)
      School results posted publicly (gross and net)
      Academically selective schools (gross and net) but no system-wide effect
      Schools practicing ability grouping (gross and net)
      One additional hour of out-of-school lessons (gross and net)
      20
      Each additional 10% of public funding(gross only)
      School principal’s perception that lack of qualified teachers hinders instruction(gross only)
      Effect after accounting for the socio-economic background of students, schools and countries
      Measured effect
      OECD (2007), PISA 2006 – Science Competencies from Tomorrow’s World, Table 6.1a
    • 43. Know how you will recognise it when you find it
      The Alchemists’ stone was to be recognisedby transforming ordinary metal into gold…
    • 44. Cross-cultural invariance of PISA
    • 45. Increased likelihood of postsec. particip. at age 19/21 associated with PISA reading proficiency at age 15 (Canada)after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group PISA Level 1)
      Odds ratioCollege entry
      School marks at age 15
      PISA performance at age 15
    • 46. Latin America then…
      Hanushek 2009
    • 47. Latin America then and now…
      Hanushek 2009
    • 48. Latin America then and now…
      Why quality is the key
      Hanushek 2009
    • 49. High policy relevance
      Quick wins
      Must haves
      Outcomes of education
      Financing of education
      Participation and progression
      Processes and structures
      Low technical quality/costly
      High technical quality
      Money pits
      Low-hanging fruits

      Low policy relevance
    • 50. Implications and lessons learned
      The medieval Alchemists’ followed the dictates of a well-established science but that was built on wrong foundations
      The search for the Holy Grail was overburdened by false clues and cryptic symbols
    • 51. Some things are always difficult
      Focus on added value of international comparisons
      Not every policy question canbebestanswered through international comparisons
      Trading off breadth and depths
      Not everything that isimportantneedstobe dealt withwithexcruciatingdetail
      Seek outputs that are as comparable as possible…
      … but as country-specific as necessary
      Focus coverage as much as feasible…
      … but keep as large as necessary to be useful for policy formation .
    • 52. High policy value
      A real-time assessment environment that bridges the gap between formative and summative assessment .
      Quick wins
      Must haves
      Examine individual, institutional and systemic factors associated with performance
      Extending the range of indicators through which educational quality is assessed
      Monitor educational progress
      Measuring growth in learning
      Low feasibility
      High feasibility
      Establish the relative standing of students and schools
      Assuming that every new indicator is orthogonal to all others
      Money pits
      Low-hanging fruits
      Low policy value
    • 53. www.oecd.org; www.pisa.oecd.org
      All national and international publications
      The complete micro-level database
      email: pisa@oecd.org
      Andreas.Schleicher@OECD.org
      … and remember:
      Without data, you are just another person with an opinion
      Thank you !

    ×