OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes - Synthesis Report: What have we learned?
Upcoming SlideShare
Loading in...5
×
 

OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes - Synthesis Report: What have we learned?

on

  • 15,477 views

Purpose: To explore how systems of E&A can be used to improve the quality, equity and efficiency of school education. ...

Purpose: To explore how systems of E&A can be used to improve the quality, equity and efficiency of school education.

Focus: A Review of national approaches to E&A in school education (primary and secondary schools)

Comprehensive approach: The Review looks at the various components of E&A such as:
Student assessment;
Teacher appraisal;
School evaluation;
The appraisal of school leaders;
Education system evaluation.

Statistics

Views

Total Views
15,477
Views on SlideShare
4,515
Embed Views
10,962

Actions

Likes
4
Downloads
199
Comments
0

13 Embeds 10,962

http://www.oecd.org 10295
http://xyofeinstein.wordpress.com 296
http://cmsnew.pdst.ie 139
http://www.pdst.ie 77
http://translate.googleusercontent.com 47
http://t4-site-mgr.oecd.org 46
http://unjobs.org 39
http://oecd.org 10
http://pdst.ie 8
http://131.253.14.98 2
http://abtasty.com 1
https://twitter.com 1
http://apertium.uoc.edu 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Teacher appraisal for performance management is most common, followed by appraisal for the completion of probation. Only 3 countries have policy frameworks for reward schemes. Probation: 15/29 Performance management: 20/29 Rewards: 3/29

OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes - Synthesis Report: What have we learned? OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes - Synthesis Report: What have we learned? Presentation Transcript

  • OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes Synthesis Report: What have we learned? Deborah Nusche, Thomas Radinger, Paulo Santiago and Claire ShewbridgeSynergies for Better Learning: An International Perspective onEvaluation and AssessmentInternational Conference, Oslo, 11-12 April 2013
  • Outline of Presentation1. Key Features of the Review2. Trends in Evaluation and Assessment (E&A)3. The evaluation and assessment framework4. Student Assessment5. Teacher Appraisal6. The Appraisal of School Leaders7. School Evaluation8. Education System Evaluation
  • 1. Key Features of the Review
  • OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes• Purpose: To explore how systems of E&A can be used to improve the quality, equity and efficiency of school education.• Focus: A Review of national approaches to E&A in school education (primary and secondary schools)• Comprehensive approach: The Review looks at the various components of E&A such as: – Student assessment; – Teacher appraisal; – School evaluation; – The appraisal of school leaders; – Education system evaluation.• Large country participation: 26 education systems / 25 countries producing a CBR; 14 country reviews.
  • OECD Review on Evaluation and Assessment Frameworks: Key issues1. Governance: Striking the right balance between central efforts and local initiative and between accountability and development2. Procedures: Designing the right instruments to ensure E&A contribute to improvement of teaching and learning3. Capacity: Developing competencies for E&A and for using feedback at all levels of the education system4. Use of results: Organising evaluative information in such a way that it facilitates effective use by stakeholders; avoiding ‘misuse’ of E&A results
  • OECD Review on Evaluation and Assessment Frameworks: Stakeholders and outputsA wide range of groups involved• National co-ordinators and informal groups within countries to produce CBRs• 28 external reviewers involved in OECD-led Review teams• About 90 schools visited and over 2 800 persons interviewed• Links with other international organisations and key stakeholder groups (BIAC, TUAC, EC, Eurydice, the World Bank, SICI, UNESCO)• Collaboration with other OECD units (PISA, TALIS, CERI’s projects, NESLI)A range of outputs• Synthesis report Outputs available at• Background papers (11) www.oecd.org/edu/evaluationpolicy• Country Background Reports (26)• Country Review Reports (14)• Qualitative data collected (29 systems / 28 countries provided data)• Meetings of the GNE on E&A (4)• International Conference to launch synthesis report in Oslo 11-12 April• Dissemination Conferences in countries
  • 2. Trends in Evaluation and Assessment
  • Trends in Evaluation and Assessment• Educational evaluation in school systems is expanding – E&A is increasingly prominent in education policy – Dedicated agencies are being created as part of new approaches to govern E&A• There is a greater variety of E&A activities• Educational measurement and indicators development are rising in importance – Student outcomes as increasingly the focal point for analysis – There is a growing emphasis on measuring student outcomes – There is a proliferation of education indicators• Larger and more varied uses are given to E&A results• Accountability as a purpose of E&A is gaining in importance• There is greater reliance on educational standards• Assessment is becoming more international• Assessment involves greater technological sophistication
  • 3. The evaluation and assessment framework: Embracing a holistic approach
  • The evaluation and assessment framework: Components1. Educational Context The educational context shapes the E&A framework2. GovernanceObjectives– Improving student outcomes! Often challenging to communicate.– Range of functions: Accountability, certification, development, diagnostic – Challenge: balance between the development and the accountability functionsResponsibilities– Shared among a wide range of agents– Dedicated intermediate agencies gain a prominent role– The devolution of responsibilities involves a variety of trade-offsIntegration of the non-public sector3. Overarching reference: Goals for student learning– General goals for education system (including equity); specific student learning objectives– Challenge: Alignment between goals for student learning and E&A
  • The evaluation and assessment framework: Components4. DesignPrinciplesPlacing students at the centre; focussing on student outcomes; committing to transparency; culture of sharing classroom practice; relying on teacher professionalism; diverse student needsMain components, Main elements within componentsArticulations: Within components; between components; moderation processes; links to classroom5. Capacity for Evaluation and AssessmentCompetencies for E&ASkills for school leadershipTools and guidelines for E&A6. Use of resultsKnowledge management– Information systems; Identification of best practices; Innovation in education;Evidence-based policy7. Implementation of evaluation and assessment policiesDivergence of views and interests; consultations; involvement of professionals; clarity of purposes; evidence to inform consensus-building; policy experimentation; evaluation of implementation; capacity building; resources; timing.
  • The evaluation and assessment framework:Practices
  • The evaluation and assessment framework:Practices
  • The evaluation and assessment framework: Policy Options1. Governance– Integrate the E&A framework, engaging in a strategic reflection– Align the E&A framework with educational goals and student learning objectives – E&A to align with the principles embedded in educational goals, be based on fit-for-purpose procedures, and rely on a clear understanding of educational goals by school agents– Secure links to the classroom and draw on teacher professionalism – Articulation of ways for the E&A framework to generate improvements in classroom practice through the E&A procedures which are closer to the place of learning– Promote national consistency while giving room for local diversity2. Design and procedures– Ensure core components are sufficiently developed within the E&A framework– Establish articulations between components of the E&A framework– Place the students at the centre of the E&A framework– Build on some key principles to effectively implement E&A – Centrality of teaching and learning; importance of school leadership; equity as a key dimension; commitment to transparency
  • The evaluation and assessment framework: Policy Options3. Capacity– Sustain efforts to improve capacity for E&A– Improve the articulation between levels of authority and assure support from the centre4. Use of results– Maintain sound knowledge management within the overall E&A framework– Commit to the use of evidence for policy development5. Implementation– Anticipate potential implementation difficulties– Engage stakeholders and practitioners in the design and implementation of E&A policies– Communicate the rationale for reform– Use pilots before full implementation and review implementation– Ensure adequate capacity and sufficient resources
  • 4. Student assessment: Putting the learner at the centre
  • Student Assessment Evaluation and assessment framework Formative and summative assessment Student Internal and external assessment assessment Classroom Teacher appraisal School 1) Trends and current School leader appraisal practices 2) The potential of studentSystem School evaluation assessment 3) Common challenges in student assessment System evaluation 4) Some policy options
  • Student Assessment: PracticesTrends1) As national curricula increasingly emphasise ‘key competencies’ or ‘21st century competencies’ for lifelong learning, countries are seeking to adapt their assessment systems in order to capture such broader types of learning.2) Research evidence on the benefits of using assessment results to inform teaching and learning has increased policy attention to formative assessment. 18/29 systems have central policy frameworks for formative assessment in place.3) Summative assessment and reporting remain important at key stages of schooling in all countries. 21/29 systems had policy frameworks for internal summative assessment in place to ensure transparency in marking and reporting.4) Concerns about the quality of learning and assessment across schools have led to a renewed focus on central standards and large-scale assessments to ensure high standards for all students. 26/37 systems have central standardised examinations in place.
  • Student Assessment: AnalysisThe potential of student assessmentFormative assessment:• Diagnose learning needs and differentiate teaching;• Provide timely feedback to students• Actively engage students in their own learning; strengthen their self-monitoring• Help teachers and students adjust teaching and learning strategiesSummative assessment:• Signal high standards and expected performance• Motivate students to increase effort and achievement• Provide information about performance to students, parents and others• Certify learning and award qualifications
  • Student Assessment: AnalysisThe challenges of student assessment• Governance: Lack of alignment between central curricula, standards and assessment approaches; lack of clarity of purposes; tensions between summative and formative functions• Procedures: Assessment formats tend to remain more traditional than curriculum goals; national assessments focus mostly on literacy and numeracy and predominantly use multiple choice and written tasks; limited use of ICT• Capacity: Limited focus on students’ own assessment competencies; inadequacies in teachers’ and school leaders’ preparation and training• Use of results: Parental concerns about inadequate reporting information; lack of transparency when using assessment results for high stakes decisions
  • Student Assessment: Policy OptionsGovernance: Establish a coherent framework for student assessment• Balance between formative and summative; internal and external assessment• Based on well-aligned reference documents (curriculum, standards, learning progressions, criteria to judge performance and exemplars illustrating different levels of achievement )• Clarity of purposeProcedures: Ensure a comprehensive approach to student assessment• Draw on a variety of assessment types to get a rounded picture of student learning• Promote assessment formats that capture valued competencies• Build on innovative approaches to assessment, and tap into the potential of ICT• Provide tools and guidelines to support effective teacher-based assessmentCapacity: Make the assessment framework participatory and build assessment competencies across the system• Build student capacity to engage in their own assessment• Promote teacher professionalism in assessment;• Develop central expertise related to student assessmentUse of results: Ensure assessment is useful and informative• Provide clear reporting guidelines to ensure transparency and fairness in reporting results• Engage parents in education through adequate reporting and communication• Promote regular use of assessment results to foster further learning
  • 5. Teacher appraisal: Enhancing teacher professionalism
  • Teacher appraisal Evaluation and assessment framework For completion of probation Student For performance assessment management Classroom Teacher appraisal For rewards School School leader appraisal 1) Trends and practicesSystem School evaluation 2) The potential of teacher appraisal 3) Common challenges in System evaluation teacher appraisal 4) Some policy options
  • Teacher Appraisal: PracticesFrameworks for teacher appraisal: trends• Teacher appraisal is the component of E&A frameworks where there is the most variation across countries• Practices range from highly prescriptive national systems to informal approaches mostly left to the school level• In many countries, there has been renewed focus on teacher appraisal in recent years, reflecting recognition that effective appraisal can contribute to improved teaching quality• Many systems (21/29) have developed central standards for the teaching profession that can guide teacher appraisal processes• Most systems (23/29) have policy frameworks for teacher appraisal in place.
  • Probation Reward scheme Country Performance management Australia Austria Belgium (Fl.) Belgium (Fr.) Canada Chile Czech Republic Denmark EstoniaSource: Information collected from countries participating in the Review Finland France Hungary Iceland Ireland Israel Italy Korea Teacher Appraisal: Practices Luxembourg Existence of policy frameworks for teacher appraisal, 2011-12 Mexico Netherlands New Zealand Norway Poland Portugal Slovak Republic Slovenia Spain Sweden UK (Northern Ireland)
  • Teacher Appraisal: AnalysisThe potential of teacher appraisal• Provide feedback to teachers, identify suitable professional development• Allow teachers to progress in their career and take on new responsibilities based on solid evaluations of their performance• Allow school leaders to be accountable for the quality of education in every classroom; move on persistently underperforming teachers
  • Teacher Appraisal: AnalysisThe challenges of teacher appraisal• Governance: High stakes teacher appraisal may lead to a climate of stress and anxiety; tensions between professional learning and accountability function of teacher appraisal• Procedures: Lack of a professional profile or standards to guide appraisal in some countries; overreliance on one or two sources of information; insufficient guidance for classroom observation; appraisal may increase workload for teachers; inadequate use of student results to evaluate teachers• Capacity: Limited preparation of teachers to benefit from their appraisal; school leaders insufficiently trained for HR management and appraisal; lack of legitimacy of evaluators• Use of results: Difficulties in ensuring systematic follow-up with professional development; absence of career opportunities for teachers; controversies around the use of appraisal results for rewards.
  • Teacher Appraisal: Policy OptionsGovernance: Establish a coherent framework for teacher appraisal• Consolidate regular developmental appraisal at the school level• Establish periodic career-progression appraisal involving external evaluators• Resolve tensions between the developmental and accountability functionsProcedures: Ensure a comprehensive approach to teacher appraisal• Establish teaching standards to guide teacher appraisal• Use multiple instruments and sources of evidence• Provide support for effective classroom observations• Avoid simplistic use of student assessment results for teacher appraisalCapacity: Build capacity of both evaluators and evaluatees• Prepare teachers for their role in the appraisal process• Strengthen the capacity of school leaders for teacher appraisal• Ensure that designated evaluators are qualified for their roleUse of results: Feed results into professional development, career advancement and accountability• Ensure that teacher appraisal feeds into professional development and school development• Establish links between teacher appraisal and career advancement decisions• Ensure that underperformance is identified and adequately addressed
  • 6. The appraisal of school leaders:Fostering pedagogical leadership in schools
  • School leader appraisal Evaluation and assessment framework Student assessment Classroom Teacher appraisal School School leader appraisal 1) Analytical approachSystem School evaluation 2) Trends and practices 3) The potential of school leader appraisal System evaluation 4) Common challenges 5) Some policy options
  • School Leader Appraisal: Analytical approachConceptualising school leadership– Impact of school leadership– The nature of effective leadership in schools– The development of new and diverse models of school leadershipScope– Appraisal of individual school leaders– Performance-management, employment-related decisions and rewards purposes
  • School leader appraisal: PracticesApproaches to the appraisal of principals in public schools1. No appraisal (6/29 countries)2. Local appraisal procedures and implementation (10/29 countries)3. Central/state requirements for appraisal (17/29 countries)– Employment-related appraisal (3/17 countries)– Mandatory periodic appraisal (14/17 countries)Existence of central/state policy frameworks (15/17 countries)1. Implementation through central/state, regional and/or intermediate authorities (9/15 countries)2. Implementation through local authorities and/or school organising bodies (9/15 countries)Objectives of appraisal1. Summative purpose (8/17 countries)2. Formative purpose (3/17 countries)3. Combination of summative and formative purposes (7/17 countries)
  • School leader appraisal: Analysis (1)The potential of school leader appraisal• Tool to communicate a vision of effective school leadership• Tool to influence school leaders’ practices and behaviours• Opportunity for reflection, feedback and support• Risk of increasing school leaders’ work load and stress levels• Concerns about existing approaches to appraisal in various countries
  • School leader appraisal: Analysis (2)Challenges for school leader appraisal• Governance: Balancing consistency through central/state frameworks with local diversity; Combining developmental and accountability functions• Procedures: Ensuring a focus on pedagogical leadership and scope for local contextualisation; Gathering a feasible amount of information to yield an accurate, fair, valid and reliable picture of performance; Accounting for student outcomes; Reflecting new and diverse models of school leadership• Capacity: Focussing attention towards the development of capacity among evaluators and school leaders• Use of results: Establishing feedback as a core element of appraisal; Ensuring links between appraisal and meaningful professional development opportunities; Establishing opportunities for career advancement
  • School leader appraisal: Policy OptionsGovernance: Promote school leader appraisal within the E&A framework• Develop a central/state policy framework to strengthen the systematic appraisal of school leaders• Ensure scope to adjust procedures to local, school and individual circumstances• Ensure links between school evaluation and school leader appraisalProcedures: Promote the appraisal of pedagogical leadership with scope for local adaptation• Develop professional standards for school leaders• Ensure scope for the local selection of appraisal aspects and criteria in line with central/state frameworks• Provide school leaders with opportunities to further develop pedagogical leadership skills• Promote the appraisal of school leaders’ competencies for monitoring, evaluation and assessmentCapacity: Build capacity of both appraisers and school leaders• Prepare school leaders and evaluators for their role in the appraisal process• Promote school leader appraisal as an opportunity for peer learningUse of results: Ensure links between appraisal and professional development, and develop career advancement opportunities• Ensure that appraisal informs meaningful professional development that pays attention to individual and school needs• Develop career advancement opportunities that are informed by appraisal procedures
  • 7. School evaluation: From compliancy to quality
  • School evaluation Evaluation and assessment framework Student assessment Classroom External evaluation Teacher appraisal (inspection / review) School School leader School self-evaluation / appraisal reviewSystem School evaluation School performance measures System evaluation
  • School Evaluation: Practices• External school evaluation is established in the vast majority of OECD countries (Except: FIN, GRC, HUN, ITA, JPN, LUX & MEX) – Typically devised by central or state authorities and conducted by Education authorities and/or specific bodies (e.g. School Inspectorate, School Review Body) – Publication of comparative school performance measures• Almost universal focus in national policy to stimulate school self-evaluation (No requirements in GRC, ITA, MEX and ESP) – Requirements vary significantly in nature, e.g. conduct self evaluation; produce specific report on school development; account for school quality – Providing comparative information to schools on performance and other measures
  • School Evaluation: Analysis - governance• Fitting school evaluation policies to the wider governance context – Public demands for school performance information – Optimising use of resources for external school evaluation (cycles, proportionality)• Articulating external school evaluation and school self- evaluation• Risks that compliancy dominates school evaluation – Multiple forms of school accountability – Demands for evidence in external evaluation / administrative burden in schools – Documentation v. classroom observation – Requirements for schools to develop specific reports
  • School Evaluation: Analysis - procedures Refence standards Instruments and information sources Administrative school reporting;COMPLIANCY Legal standards Verification during external school evaluation visitsQUALITY: Criteria to evaluate educational Classroom observation - and evaluationEducational processes, notably teaching and indicators; Mechanisms for feedback frompractices learning students, teachers and parents School-developed tests to monitor studentOutcomes Student learning objectives progress; Standardised assessments /examinations School-developed surveys, student and Student, parent and staff school councils/ centrally-developed surveys satisfaction or questionnaires as part of standardised assessmentsSchool capacity School leadership; Effectiveness of Verification of school self-evaluationfor self- school self-evaluation; processes and results during external schoolevaluation School capacity to improve evaluation visits
  • School Evaluation: Analysis - capacity Related challenges for capacity Ensuring legal knowledge/competencies within the external school evaluationCOMPLIANCY body and school leadership; Reducing the burden of compliancy reporting on schoolsQUALITY: Recruitment and training of external evaluators/reviewers;Educational Training school leadership in undertaking classroom observation;practices Stimulating and supporting peer reviews among schools; Central support tools for stakeholder surveys; Engaging teachers in analysis of student results and giving them autonomy to make changes for improvement;Outcomes Time and resources at the school level to analyse data Central capacity to develop suitable standardised assessments and capacity to report back results Ensuring leadership of school self-evaluation activities: Building trust in self-evaluation processes;School capacity for Engaging school leaders in external school evaluation activities;self-evaluation Creating roles and responsibilities within the school; Engaging the full community in school self-evaluation activities
  • School Evaluation: Analysis – use of results Related challenges for the reporting and use of results Separating school audit and school self-evaluation reporting;COMPLIANCY Sanctions for schools not complying with legal standardsQUALITY: Reporting of results of external school evaluation; Immediate feedback to teachers and school leaders;Educational Feeding into professional development and school developmentpractices Mechanisms to follow up how schools act on external school evaluation results when educational practices are identified for improvement School developed tests: Integrating these results into analysis of school development priorities, including professional development needs Standardised assessments/ examinations: Timeliness of feedback of data;Outcomes validity and relevance of data; tools for analysis; accuracy of reporting and contextualisation of results Integration of student, staff and parent satisfaction results into school reporting and development plans School reporting to its community; Integrating the use of self-evaluationSchool capacity for results in school development/strategic improvement cycles;self-evaluation School capacity and the cycle/intensity of external school evaluations
  • School Evaluation: Policy OptionsGovernance: Underlies all options on procedures, capacity and use of results• Clarify the role and purpose of school evaluation• Focus on the improvement of teaching, learning and student outcomes• Evaluate and adapt external school evaluation to reflect the maturity of the school evaluation culture• Raise the profile of school self-evaluationProcedures: Ways to minimise the burden and maximise the fairness• Develop nationally agreed criteria for school quality to guide school evaluation• Develop appropriate resources for school self-evaluation• Ensure transparency in external school evaluation procedures and a strong evidence baseCapacity: A priority for school improvement• Ensure credibility of external school evaluators and sufficient capacity and retraining as necessary• Strengthen school principals’ capacity to stimulate an effective school self-evaluation culture• Promote the engagement of all staff and students in school self-evaluation• Promote peer learning among schoolsUse of results: Learning what to improve and where to intervene• Promote wider use of external school evaluation results and ensure systematic follow up• Optimise feedback of centrally collected data to schools for self-evaluation and development planning• Report a broad set of school performance measures with adequate contextual information
  • School Evaluation: Policy OptionsGovernance: Underlies all options on procedures, capacity and use of results• Clarify the role and purpose of school evaluation• Focus on the improvement of teaching, learning and student outcomes• Evaluate and adapt external school evaluation to reflect the maturity of the school evaluation culture• Raise the profile of school self-evaluationProcedures: Ways to minimise the burden and maximise the fairness• Develop nationally agreed criteria for school quality to guide school evaluation• Develop appropriate resources for school self-evaluation• Ensure transparency in external school evaluation procedures and a strong evidence baseCapacity: A priority for school improvement• Ensure credibility of external school evaluators and sufficient capacity and retraining as necessary• Strengthen school principals’ capacity to stimulate an effective school self-evaluation culture• Promote the engagement of all staff and students in school self-evaluation• Promote peer learning among schoolsUse of results: Learning what to improve and where to intervene• Promote wider use of external school evaluation results and ensure systematic follow up• Optimise feedback of centrally collected data to schools for self-evaluation and development planning• Report a broad set of school performance measures with adequate contextual information
  • 8. Education system evaluation: Informing policies for system improvement
  • Education system evaluation Evaluation and assessment framework Student assessment Classroom Teacher appraisal School School leader appraisal National education systemSystem School evaluation Sub-national education system e.g. local authority, schools System evaluation in religious/pedagogical network
  • Education System Evaluation: Practices • Major increase in use of monitoring systems across OECD – Large-scale student assessments • By the late 1990s all OECD countries had participated in an international study • Development of national assessments – Stakeholder surveys and longitudinal information A A B B C C C D E F F H I I I I K L M N N N P P S S E U U F F A H Z N S I R U S R S T O U E L Z O O R V V S SW UK- S T L R N L E K T N A N L L R A R X X D L R L T N K P E NINat Asst: FullcohortNat Asst:SampleSurvey:StudentsSurvey:TeachersSurvey: ParentsLongitudinalinformation • Thematic evaluations in samples of schools
  • Education System Evaluation: Analysis - governance • Calls to monitor performance in the public sector • Recognising the economic importance of education • Many systems do not have an overall framework for education system evaluationIndicators of a strategic approach Countriesinformation collectionMapping against system priorities and Australia; Czech Republic; Hungary; Israel;plan to prioritise new collection Netherlands; Slovak RepublicMapping against system priorities France; Iceland; Ireland; Northern Ireland (UK)Plan to prioritise collection of new Belgium (French & Flemish Comm.); Chile;information Finland; Slovenia; SpainNeither Austria; Denmark; Italy; Korea; Luxembourg; Mexico; New Zealand; Norway; Poland; Sweden
  • Education System Evaluation: Analysis - procedures• Reference standards may be many – Broad goals for education system; Student learning objectives / standards; Objectives set for specific education policies – The use of targets for system performance • E.g. European Union benchmarks • Improvement or decline over time in specific assessments• Monitoring trends in equity and quality – Availability and reliability of contextual information – Prioritising reliability of national assessments, but validity? • Multiple choice (27); Closed-format short answer (21); open-ended writing tasks/calculations (17); Performing a task (7); oral elements (6) • Broader curriculum coverage with sample surveys v full cohort – Capitalising on technology – Tensions: monitoring trends and holding schools accountable – Specific reviews on aspects of the school system
  • Education System Evaluation: Analysis - capacity• Capacity to produce evidence on system performance and use results – The creation of specific bodies – Credibility and objectivity of evidence and reporting underlying policy development – Capacity to synthesise and disseminate results• Evaluation capacity at sub-national level may vary significantly – Local monitoring and school use of data • Aligning goals • Building trust
  • Education System Evaluation: Analysis – reporting and use of results• Communicating education system evaluation results clearly and comprehensively – Reporting principles: relevance; credibility; timeliness; accessibility and interpretability – Avoiding media misinterpretation – Engaging key stakeholders in discussion of results• Making better use of results in planning and policy development – Too much information! Findings ways to better channel results – Overall summative report on the education system – Political urgency versus availability of broad research base – Research and empirical analysis – Information systems to promote use of results at local levels
  • Education System Evaluation: Policy OptionsGovernance: Being systematic and strategic for better informed policy making• Ensure a broad concept of education system evaluation within the E&A framework• Ensure policy making is informed by high-quality measures, but not driven by their availability• Situate education system evaluation in the broader context of public sector performance requirementsProcedures: Developing an approach to learn from a broad evidence base• Develop a national education indicator framework• Design a national strategy to monitor student learning standards• Ensure the collection of: qualitative information; and contextual information to monitor equity• Assure the monitoring of changes over time and progress of particular student cohortsCapacity: Ensuring continuity and credibility• Establish and secure capacity for education system evaluation• Promote the development of evaluation capacity at the local authority level• Ensure objectivity and credibility in education system evaluation activitiesUse of results: Strengthening analysis for system improvement• Strengthen analysis of education system evaluation results for planning and policy development• Communicate key results of education system evaluation to stakeholders• Support feedback for local monitoring
  • Thank you for your attention! www.oecd.org/edu/evaluationpolicy