Successfully reported this slideshow.
OECD REVIEW ON EVALUATION AND           ASSESSMENT FRAMEWORKS FOR           IMPROVING SCHOOL OUTCOMESDeborah Nusche, Analy...
Key topics for this presentation• Why do evaluation and assessment matter?• The OECD project• Key findings from the Review
Why do evaluation and assessment matter?• Highest rated education policy priority among OECD countries• Analysing strength...
OECD Review on Evaluation and Assessment        Frameworks for Improving School Outcomes• Purpose: To explore how systems ...
OECD Review on Evaluation and AssessmentFrameworks: Overview   Evaluation and assessment framework           Student asses...
OECD Review on Evaluation and Assessment     Frameworks: MethodologyAnalytical strand    – Reviewing the literature and ev...
OECD Review on Evaluation and Assessment     Frameworks: Participation• 28 systems are providing detailed data on their ev...
OECD Review on Evaluation and Assessment     Frameworks: Outputs• Background papers on issues such as formative  assessmen...
OECD Review on Evaluation and Assessment      Frameworks: Key topics1. Governance: Striking the right balance between cent...
OECD Review on Evaluation and Assessment      Frameworks: GovernanceAchieving the right balance between central and local ...
OECD Review on Evaluation and Assessment                        Frameworks: Governance                               Chart...
OECD Review on Evaluation and Assessment       Frameworks: GovernanceAdvantages of a decentralised approach:•   Evaluation...
OECD Review on Evaluation and Assessment      Frameworks: GovernanceSome suggestions for policy development• Develop centr...
OECD Review on Evaluation and Assessment      Frameworks: ProceduresUsing the right procedures for evaluation and assessme...
OECD Review on Evaluation and Assessment  Frameworks: ProceduresTesting formats used in national assessments, 2012        ...
OECD Review on Evaluation and Assessment      Frameworks: ProceduresSome suggestions for policy development• Ensure that a...
OECD Review on Evaluation and Assessment      Frameworks: CapacityHow to ensure that stakeholders have the right competenc...
OECD Review on Evaluation and Assessment                  Frameworks: CapacityPercentage of teachers in lower secondary ed...
OECD Review on Evaluation and Assessment         Frameworks: CapacitySome suggestions for policy development•   Build stud...
OECD Review on Evaluation and Assessment      Frameworks: Use of results• Evaluation and assessment are typically used for...
20                                                               30                                                       ...
OECD Review on Evaluation and Assessment      Frameworks: Use of resultsSome suggestions for policy development• Ensure th...
VIELEN DANK!MEHR INFORMATIONENWWW.OECD.ORG/EDU/EVALUATIONPOLICY
Upcoming SlideShare
Loading in …5
×

OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes - Deborah Nusche, Analyst, OECD Directorate for Education - Bundesministerium für Unterricht, Kunst und Kultur Vienna, 4 December 2012

2,222 views

Published on

Why do evaluation and assessment matter?

Highest rated education policy priority among OECD countries
Analysing strengths and weaknesses of the education system is key to defining strategies for further development
Need to provide information about education quality to parents and society at large
Decentralisation and school autonomy are creating a greater need for evaluation and assessment
To ensure that schools get the information and feedback they need to improve their work
To allow the government to monitor if national goals for quality and equity in education are achieved

Published in: Education
  • Be the first to comment

OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes - Deborah Nusche, Analyst, OECD Directorate for Education - Bundesministerium für Unterricht, Kunst und Kultur Vienna, 4 December 2012

  1. 1. OECD REVIEW ON EVALUATION AND ASSESSMENT FRAMEWORKS FOR IMPROVING SCHOOL OUTCOMESDeborah Nusche, AnalystOECD Directorate for EducationBundesministerium für Unterricht, Kunst und KulturVienna, 4 December 2012
  2. 2. Key topics for this presentation• Why do evaluation and assessment matter?• The OECD project• Key findings from the Review
  3. 3. Why do evaluation and assessment matter?• Highest rated education policy priority among OECD countries• Analysing strengths and weaknesses of the education system is key to defining strategies for further development• Need to provide information about education quality to parents and society at large• Decentralisation and school autonomy are creating a greater need for evaluation and assessment – To ensure that schools get the information and feedback they need to improve their work – To allow the government to monitor if national goals for quality and equity in education are achieved
  4. 4. OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes• Purpose: To explore how systems of evaluation and assessment can be used to improve the quality, equity and efficiency of school education.• Focus: A Review of national approaches to evaluation and assessment in school education (primary and secondary schools)• Comprehensive approach: The Review looks at the various components of assessment and evaluation such as: – Student assessment; – Teacher appraisal; – School evaluation; – Education system evaluation.
  5. 5. OECD Review on Evaluation and AssessmentFrameworks: Overview Evaluation and assessment framework Student assessment Classroom Teacher appraisal Improvement of teaching and School learning outcomes School evaluation System System evaluation
  6. 6. OECD Review on Evaluation and Assessment Frameworks: MethodologyAnalytical strand – Reviewing the literature and evidence on the impact of evaluation and assessment procedures – Gathering data on countries’ policies and practicesCountry Review strand – Country Reviews provide specific advice to individual countries. – OECD-led Review Team – The scope and focus is determined by the country in consultation with the SecretariatSynthesis report – Comparative report to analyse policy options and highlight good practices across countries.Spotlight reports – Short reports providing a customised summary of the synthesis report putting the spotlight on a particular country
  7. 7. OECD Review on Evaluation and Assessment Frameworks: Participation• 28 systems are providing detailed data on their evaluation and assessment practices• 26 systems are preparing a comprehensive Country Background Report• 14 systems (in green) opted for a Country Review• 6 systems (in orange) opted for a Spotlight Report Australia, Austria, Belgium (Flemish Community), Belgium (French Community), Canada, Chile, Czech Republic, Denmark, Finland, France, Hungary, Iceland, Ireland, Italy, Korea, Luxembourg, Mexico, Netherlands, New Zealand, Norway, Poland, Portugal, Slovak Republic, Slovenia, Sweden and United Kingdom (Northern Ireland).
  8. 8. OECD Review on Evaluation and Assessment Frameworks: Outputs• Background papers on issues such as formative assessment, use of test results, teacher appraisal, school evaluation etc. (11)• Country Background Reports (26)• OECD Publication series: OECD Reviews of Evaluation and Assessment in Education (14 Country Review Reports)• Final Synthesis Report (early 2013)All outputs available at:www.oecd.org/edu/evaluationpolicy
  9. 9. OECD Review on Evaluation and Assessment Frameworks: Key topics1. Governance: Striking the right balance between central efforts and local initiative and for improvement2. Procedures: Designing the right instruments to ensure evaluation and assessment contribute to improvement of teaching and learning3. Capacity: Developing competencies for evaluation and for using feedback at all levels of the education system4. Use of results: Organising evaluative information in such a way that it facilitates effective use by stakeholders; avoiding ‘misuse’ of E&A results
  10. 10. OECD Review on Evaluation and Assessment Frameworks: GovernanceAchieving the right balance between central and local initiatives• Decentralisation and school autonomy are widespread across OECD countries• In about two-thirds of OECD countries, most schooling decisions in lower secondary education are taken at the local or school level• At the same time: Central authorities remain responsible for ensuring quality and equity of education nationally• Strong accountability mechanisms are often seen as a counterbalance to school autonomy
  11. 11. OECD Review on Evaluation and Assessment Frameworks: Governance Chart D6.1 Percentage of decisions taken at each level of government in public lower secondary education (2011) School Local 100 90 80 70 60 50 40 30 20 10 0 Australia Turkey Scotland Germany Italy Austria England Spain Estonia Iceland Greece Hungary Portugal Indonesia Poland Slovenia Chile Israel France Japan Canada Ireland Belgium (Fl.) Denmark Korea United States Sweden Switzerland Luxembourg Mexico Netherlands Belgium (Fr.) Norway Slovak Republic OECD average Czech RepublicCountries are ranked in descending order of the percentage of decisions taken at the school level.Source: OECD, Education at a Glance 2012. Table D6.1 See Annex 3 for notes (www.oecd.org/edu/eag2012).
  12. 12. OECD Review on Evaluation and Assessment Frameworks: GovernanceAdvantages of a decentralised approach:• Evaluation and assessment will only lead to improvement if stakeholders are fully engaged and use results / feedback to adjust their practices• Ownership and responsibility for evaluation and assessment can generate commitment, professionalism and dynamism.• Diversity of approaches allows for innovation.Challenges of a decentralised approach• Not all sub-national authorities and schools are systematic in their evaluation activities  Concerns about those regions & schools that have less capacity or commitment for implementing quality management approaches.• School practices in student assessment, teacher appraisal and school self-evaluation are variable and often depend on the leadership of individual principals and teachers.  Concerns about equity and equivalence of education across the country• The potential for system learning is not fully exploited.
  13. 13. OECD Review on Evaluation and Assessment Frameworks: GovernanceSome suggestions for policy development• Develop central reference points for common orientation – Student assessment: Standards, learning progressions, assessment criteria and exemplars to support fair and reliable assessment of students across schools – Teacher appraisal: A professional profile or standards for the teaching profession as a basis for initial teacher education, teacher appraisal and professional development – School evaluation: Shared quality indicators to guide both school self-evaluation and external evaluations• Map out existing tools to support student assessment, teacher appraisal and school evaluation• Clarify responsibilities for different aspects of evaluation and assessment• Follow up with competency descriptions• Develop adequate professional development offers
  14. 14. OECD Review on Evaluation and Assessment Frameworks: ProceduresUsing the right procedures for evaluation and assessment• Research shows that evaluation and assessment can have a positive or negative impact on teaching and learning, depending on how approaches are designed and implemented• Key question: How to design evaluation and assessment in a way that optimises potential positive effects and minimise potential negative effects?Some challenges• Aspects assessed: How to avoid signalling narrow expectations?• Instruments used: How to find the right mix of instruments to give attention to all valued aspects of performance ? How to optimise use of ICT?• Adaptations to context: How to ensure equity and fairness in E&A?
  15. 15. OECD Review on Evaluation and Assessment Frameworks: ProceduresTesting formats used in national assessments, 2012 Closed-format Open-ended writing Performing Oral questions and Multiple choice short answer Oral presentation tasks/calculations a task answers questions Austria Yes Yes Yes No Yes Yes Australia Yes Yes Yes No No No Belgium (Fr.) Yes Yes Yes Yes No No Belgium (Fl.) Yes Possible Possible Possible No No Czech Republic Yes Yes No No No No Denmark Yes Yes No No No No Finland Yes Yes No No No No France Yes Yes No No No No Hungary Yes No No Vocational only Vocational only No Iceland Yes No ISCED 2 only ISCED 1 only ISCED 2 only No Ireland Yes Yes No No No No Israel Yes Yes Yes No No No Italy Yes Yes No No No NoMexico (sample) Yes Yes Yes No No No Mexico (full) Yes No No No No No Netherlands Yes No Yes No No No New Zealand Yes Yes Yes Yes Yes No Norway Yes Yes No No No No Poland Yes No Yes No No No Slovenia Yes No Yes No No No Sweden Yes Yes Yes ISCED 1 only Yes ISCED 3 only
  16. 16. OECD Review on Evaluation and Assessment Frameworks: ProceduresSome suggestions for policy development• Ensure that all important aspects of teaching and learning are given some attention in evaluation and assessment (beyond basic knowledge and skills)• Draw on a comprehensive set of evaluation and assessment methodologies (observation, performance-tasks, dialogue, portfolios)• Use multiple evaluators and instruments, both internal and external to the school• Explore the potential of ICT to facilitate innovative evaluation and assessment methods• Ensure that the context is considered in implementing evaluation and assessment procedures
  17. 17. OECD Review on Evaluation and Assessment Frameworks: CapacityHow to ensure that stakeholders have the right competencies to evaluate anduse evaluation results?• Increasing focus on the role of the learner in assessment – Need to build student competencies to become active participants in their own assessment – Teachers themselves need to develop skills for learner-centred teaching and assessment• Countries report gaps in teacher skills regarding assessment – Attention to assessment varies across initial teacher education institutions – Approaches to professional development in assessment are often fragmented• School leaders may not be well prepared for leading evaluation and assessment in their schools – In many countries they have traditionally more administrative roles – Lack of training regarding analysis of assessment data, teacher appraisal and school self- evaluation
  18. 18. OECD Review on Evaluation and Assessment Frameworks: CapacityPercentage of teachers in lower secondary education who have a ‘high level of need’ ofprofessional development in student assessment practices 50 45 40 35 30 25 20 15 10 5 0 Mexico Ireland Denmark Bulgaria Italy Belgium (Fl.) Australia TALIS Average Poland Norway Hungary Korea Malaysia Brazil Turkey Iceland Austria Malta Spain Slovenia Lithuania Slovak Republic PortugalSource: OECD TALIS Database Estonia
  19. 19. OECD Review on Evaluation and Assessment Frameworks: CapacitySome suggestions for policy development• Build student capacity for self-assessment – Students will only develop their assessment capacity if teachers themselves have such capacity and are adequately supported• Build teacher capacity for student assessment – Priority areas may include: assessment of key competencies; reliable grading; effective formative assessment, inclusive and fair assessment of diverse students; use of national test results – Adopt a strategic approach to teacher learning in assessment throughout initial teacher education and professional development• Build the capacity of school leaders for teacher appraisal and school self-evaluation – In particular the capacity to evaluate, coach and guide staff; use of data for improvement.• Build the capacity of sub-national authorities for school and local system evaluation – In particular the capacity to understand and make decisions based on data; to provide constructive feedback to schools• Build central expertise for system evaluation
  20. 20. OECD Review on Evaluation and Assessment Frameworks: Use of results• Evaluation and assessment are typically used for both improvement and accountability purposes• But countries vary in the degree to which they emphasise one or the other – Countries with a strong focus on improvement typically emphasise: formative, low- stakes assessment of students; teacher appraisal that is linked to decisions regarding teacher professional development and learning opportunities; school self-evaluation and external support for organisational learning. – Countries with a strong focus on accountability typically emphasise: high-stakes standardised assessment of students; teacher appraisal that is linked to decisions regarding career advancement, salary, promotion and dismissal; external reviews or inspections of school quality; publication of school evaluation results and / or public comparisons of school performance.• Where the accountability function overshadows the improvement function, this may lead to strategic responses (focussing only on the areas that are assessed)
  21. 21. 20 30 50 60 70 90 10 40 80 0 % United States United Kingdom New Zealand Netherlands Slovak Republic SwedenSource: OECD PISA Database . Norway Canada Poland Turkey Australia Denmark Luxembourg OECD average achievement data was posted publicly (2009) Slovenia Chile Mexico Frameworks: Use of results Hungary Korea Estonia Greece Czech Republic Italy Portugal Israel Iceland Ireland OECD Review on Evaluation and Assessment Germany Percentage of 15-year-old students in schools where the principal reported that student Spain Austria Japan Switzerland Finland Belgium
  22. 22. OECD Review on Evaluation and Assessment Frameworks: Use of resultsSome suggestions for policy development• Ensure the results of evaluation and assessment are used to make further improvements to teaching and learning – this should be the key purpose of all evaluation and assessment activities• Link evaluation and assessment activities to professional learning and school development• Use multiple sources of evidence when using results for summative / accountability purposes• Consider the potential unintended effects when establishing high stakes accountability systems• Ensure that results are used in line with the original purpose of each instruments. Avoid misuse of results.
  23. 23. VIELEN DANK!MEHR INFORMATIONENWWW.OECD.ORG/EDU/EVALUATIONPOLICY

×