Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

From assessment to action


Published on

From assessment to action: Impact of student assessment data on educational policy reform for sustainable future.
Autors: Laura Paviot and Mioko Saito, from IIEP-UNESCO

Published in: Government & Nonprofit
  • Be the first to comment

From assessment to action

  1. 1. CIES 2016 Vancouver, Canada 9 March 2016 From assessment to actions: Impact of student assessment data on educational policy reform for sustainable future Learning for All: Using assessment data for policy and planning in Asia Laura Paviot and Mioko Saito, IIEP
  2. 2. Since 1994 Collaboration with SACMEQ 1989-1994 NLSA Zimbabwe 2000-2003 NLSA Viet Nam Since 2014 Learning Portal, e-Forum Since 2014 Collaboration with NEQMAP & SEA-PLM Since 2010 Collaboration with PASEC Background of this panel 1969 / 1978 / 1989 Seminars on QoE 2004 Policy Forum on C-N Ass’t 2016 MOOC Since 1991 Specialized Training Course 1969 / 1978 / 1989 Seminars on QoE Since 2014 Collaboration with NEQMAP & SEA-PLM Since 2010 Collaboration with PASEC
  3. 3. Presentation Outline • Trends in learning assessments • Researchers vs. policy-makers • “Enlightenment model” as research utilization • Categories of actions from assessment evidence • Illustrations: Vietnam and South Korea • MOOC survey
  4. 4. Frequency of National Assessments: Pre and Post Dakar Source: UNESCO GMR (2015)
  5. 5. Learning Assessment Capacity Index (LACI) 2010-2015 Source: UIS (2016) National, primary, 2010-2015 National, secondary, 2010-2015 Trend data for primary & secondary, 2010-2015 International assessment, 2010-2015 Regional assessment, 2010-2015
  6. 6. Two “communities”: Policy Makers vs. Researchers Policy Makers Researchers Problem Definition Deal with complex real world social problems Develop questions from theoretical frameworks Culture Targets, ways and means Generalization and explanation “what” results “how” and “why” results Use procedural and legal jargon Use restricted technical senses Role & Accountability Risk-avoidance New and unexpected Time frame Quick solutions Need time – internal quality criteria Source: Cross et al (2000)
  7. 7. Policy research cycle Interpretation and reporting Data collection and analysis Specific research questions Programme implementation Source: Saito (1999)
  8. 8. Enlightenment Model • Major way with research effect on policy • Role of researchers and educational planners as “information brokers” • Research ‘enlightens’ policy makers • Findings made available by different channels • Sensitize and inform public opinion • Provide orientation for reflection • Ideals and evidence with ‘percolating’ effect Source: Weiss (1979); Postlethwaite (2001); Ross et al (2006)
  9. 9. Enlightened policy research cycle Policy reform and agenda for action Consultation and debate Policy suggestions Interpretation and reporting Data collection and analysis Specific research questions General policy concerns Programme implementation Source: Saito (1999) Mass media, journals, conference Sensitize public Orientation to expand different reflections Feed debate Research Knowledge
  10. 10. Categories of ‘change’ after assessments • General analyses of education system • Inputs to a general review of policy • Preparation of a reform programme • Dissemination and discussion of results with stake-holders • Reform of the intended curriculum • Reform of the implemented curriculum • Improvement and reallocation of resources • Monitoring of the effects of an intervention • Helping donors identify aspects of the education system • Improvement of learning achievement Source: Postlethwaite and Kellaghan (2008)
  11. 11. Vietnam • Assessments – National assessment, Grade 5, 2001, 2007, 2011 – Expanded to lower and upper secondary levels since 2011 – Participation in PISA 2012 and PASEC 2013 • Findings – Disparity (location, SES, ethnicity) • Research Utilization and Actions – Until 2010 no data usage on policy – Targeted resource allocation for disadvantaged population – Curriculum and textbook reform to develop cognitive and non- cognitive skills – Re-orient teaching methods to apply knowledge in “real life” situation Source: Le (2014)
  12. 12. South Korea • Assessments – Since 1995, TIMSS, PISA, ICILS to establish benchmark – National assessment since 2008 to identify underachievers • Findings – Top cognitive results and bottom affective results in ILSA – Disparity (gender, location) in NLSA • Research Utilization and Actions – Recommendations from ILSA rarely used – ‘Targeted programmes’ based on NLSA results – Research competition on ideas on using ILSA & NLSA results by policy makers Source: Cho (2014)
  13. 13. IIEP MOOC Survey • February 2016 IIEP MOOC on Learning Assessments • 2,700 registered (F 53%; A&P 36%); 1,000 active; • 295 responses (F 54%; A&P 36%) as of 8 March • Questions on: –(i) how well informed on results; –(ii) opportunities to participate; and –(iii) perceived changes in inputs, policy review, reforms, etc.
  14. 14. Preliminary Results on IIEP Survey • Much more informed on public examinations and NLSA (rather than ILSA and RLSA) • Much more informed about performance than on its enabling variables • Information sources – (1) Official documents; (2) Observation; (3) Informal conversation • Opportunity for involvement – (1) questionnaire construction; (2) item writing • Perceived changes after assessments – (1) learning improvement; (2) general analyses • Policy-research process as extra work as ‘leaders’; not teachers and school heads
  15. 15. Conclusion from country illustrations • Possible to observe: – How research was used – Category of action • Illustration of “enlightenment model” – Dissemination via different channels – Active participation of actors • Resulted in the improvement of student achievement