From assessment to action: Impact of student assessment data on educational policy reform for sustainable future. Autors: Laura Paviot and Mioko Saito, from IIEP-UNESCO

1. CIES 2016
Vancouver, Canada
9 March 2016
From assessment to actions: Impact of
student assessment data on educational
policy reform for sustainable future
Learning for All: Using assessment data for policy and
planning in Asia
Laura Paviot and Mioko Saito, IIEP

2. Since 1994 Collaboration with SACMEQ
1989-1994
NLSA Zimbabwe
2000-2003
NLSA Viet Nam
Since 2014
Learning Portal, e-Forum
Since 2014
Collaboration with
NEQMAP & SEA-PLM
Since 2010
Collaboration with PASEC
Background of this panel
1969 / 1978 / 1989
Seminars on QoE
2004 Policy Forum
on C-N Ass’t
2016 MOOC
Since 1991 Specialized Training Course
1969 / 1978 / 1989
Seminars on QoE
Since 2014
Collaboration with
NEQMAP & SEA-PLM
Since 2010
Collaboration with PASEC

3. Presentation Outline
• Trends in learning assessments
• Researchers vs. policy-makers
• “Enlightenment model” as research utilization
• Categories of actions from assessment
evidence
• Illustrations: Vietnam and South Korea
• MOOC survey

4. Frequency of National Assessments:
Pre and Post Dakar
Source: UNESCO GMR (2015)

5. Learning Assessment Capacity Index (LACI)
2010-2015
Source: UIS (2016)
National, primary, 2010-2015
National, secondary, 2010-2015
Trend data for primary & secondary, 2010-2015
International assessment, 2010-2015
Regional assessment, 2010-2015

6. Two “communities”:
Policy Makers vs. Researchers
Policy Makers Researchers
Problem
Definition
Deal with complex real
world social problems
Develop questions from
theoretical frameworks
Culture Targets, ways and means Generalization and
explanation
“what” results “how” and “why” results
Use procedural and legal
jargon
Use restricted technical
senses
Role &
Accountability
Risk-avoidance New and unexpected
Time frame Quick solutions Need time – internal
quality criteria
Source: Cross et al (2000)

7. Policy research
cycle
Interpretation
and reporting
Data
collection
and analysis
Specific
research
questions
Programme
implementation
Source: Saito (1999)

8. Enlightenment Model
• Major way with research effect on policy
• Role of researchers and educational planners
as “information brokers”
• Research ‘enlightens’ policy makers
• Findings made available by different channels
• Sensitize and inform public opinion
• Provide orientation for reflection
• Ideals and evidence with ‘percolating’ effect
Source: Weiss (1979); Postlethwaite (2001); Ross et al (2006)

9. Enlightened policy
research cycle
Policy reform and
agenda for action
Consultation
and debate
Policy
suggestions
Interpretation
and reporting
Data
collection
and analysis
Specific
research
questions
General
policy
concerns
Programme
implementation
Source: Saito (1999)
Mass
media,
journals,
conference
Sensitize
public
Orientation to
expand
different
reflections
Feed
debate
Research
Knowledge

10. Categories of ‘change’ after assessments
• General analyses of education system
• Inputs to a general review of policy
• Preparation of a reform programme
• Dissemination and discussion of results with stake-holders
• Reform of the intended curriculum
• Reform of the implemented curriculum
• Improvement and reallocation of resources
• Monitoring of the effects of an intervention
• Helping donors identify aspects of the education system
• Improvement of learning achievement
Source: Postlethwaite and Kellaghan (2008)

11. Vietnam
• Assessments
– National assessment, Grade 5, 2001, 2007, 2011
– Expanded to lower and upper secondary levels since 2011
– Participation in PISA 2012 and PASEC 2013
• Findings
– Disparity (location, SES, ethnicity)
• Research Utilization and Actions
– Until 2010 no data usage on policy
– Targeted resource allocation for disadvantaged population
– Curriculum and textbook reform to develop cognitive and non-
cognitive skills
– Re-orient teaching methods to apply knowledge in “real life”
situation
Source: Le (2014)

12. South Korea
• Assessments
– Since 1995, TIMSS, PISA, ICILS to establish benchmark
– National assessment since 2008 to identify underachievers
• Findings
– Top cognitive results and bottom affective results in ILSA
– Disparity (gender, location) in NLSA
• Research Utilization and Actions
– Recommendations from ILSA rarely used
– ‘Targeted programmes’ based on NLSA results
– Research competition on ideas on using ILSA & NLSA
results by policy makers
Source: Cho (2014)

13. IIEP MOOC Survey
• February 2016 IIEP MOOC on Learning
Assessments
• 2,700 registered (F 53%; A&P 36%); 1,000 active;
• 295 responses (F 54%; A&P 36%) as of 8 March
• Questions on:
–(i) how well informed on results;
–(ii) opportunities to participate; and
–(iii) perceived changes in inputs, policy review,
reforms, etc.

14. Preliminary Results on IIEP Survey
• Much more informed on public examinations and NLSA
(rather than ILSA and RLSA)
• Much more informed about performance than on its
enabling variables
• Information sources – (1) Official documents; (2)
Observation; (3) Informal conversation
• Opportunity for involvement – (1) questionnaire
construction; (2) item writing
• Perceived changes after assessments – (1) learning
improvement; (2) general analyses
• Policy-research process as extra work as ‘leaders’; not
teachers and school heads

15. Conclusion from country illustrations
• Possible to observe:
– How research was used
– Category of action
• Illustration of “enlightenment model”
– Dissemination via different channels
– Active participation of actors
• Resulted in the improvement of student
achievement