This document discusses using evidence to evaluate quality and equity in education. It presents examples of evidence that could be used and questions about how key players might interpret and apply the data. It then outlines challenges around generating demand for evidence among decision-makers and ensuring its proper use in policymaking, governance, and school management. Specifically, it addresses how to determine relevant evidence needs, interpret data through research, communicate findings clearly, and avoid potential traps like over-reliance on metrics or ignoring complex realities.

1. Evidence of Quality and
Equity in Education
Péter Radó
Belgrade, 06.12.2011.

2. A sample evidence 1.

3. A sample evidence 2.

4. A sample evidence 3.

5. A sample evidence 4.

6. A few questions:
(Assuming that these data are available )
• Is there a demand for these data among the key players of education?
• Do you have access to the information allowing the interpretation of
these data? What would it tell about the quality of educational services?
• Do you have the capacity to transform the conclusions drawn from these
data to actual design of policies and implementation strategies?
• Can you imagine a minister of education explaining this statistical
analysis in a 30 seconds TV interview?
• Can you imagine convincing middle class parents about mixing their
children with disadvantaged children on the basis of this analysis?
• Can you imagine convincing teachers about forming heterogeneous
classrooms instead of forming easily taught homogeneous classrooms?
• Can you imagine municipalities reorganizing their local school systems
in order to ensure equal distribution of disadvantaged students on the
basis of this analysis?

7. Questions emerging from the questions
1. What can generate demand for evidence?
2. How to produce evidence?
3. How to determine quality and equity relevant
evidence needs?
4. How to ensure the interpretation of evidence?
5. What are the traps to be avoided?
6. What is the best use of evidence? (Influencing the
quality of the policy discourse.)

8. Generating demand for evidence
From dumb centralization to intelligent post-bureaucratic
governance

9. Generating demand for evidence:
governance and management
Making education a complex adaptive system by:
• making schools autonomous learning organizations
accountable for the quality of their services and the equity
of their outcomes,
• decentralization, deregulation, procedural regulation,
• mandatory multilevel short- and medium term planning,
• Intelligent professional accountability systems with strong
focus on educational outcomes produced by whole schools,
• ensuring the openness of decision making at all levels.

10. Generating demand for evidence:
policy-making
The normative approach to
EBPM ↔ the very
contextual nature of quality
and equity →
„Intelligent policy making”
Learning by open
deliberation: enrichment of
reasoning with the tacit and
practical knowledge of
practitioners + evidence
informing the actors of the
policy discourse

11. Information production in education 1.
Quality evaluation embedded to a
performanace management
system
• Connecting goals/targets with
quality evaluation instruments
• Student performance assessment
informing (external and internal)
school evaluation
• Connecting the pillars of by quality
evaluation by an integrated
indicator/reporting system
• Strengthening accountability by
intervention on the basis of quality
evaluation information
• Balancing accountability and
organisational learning in schools

12. Information production in education 2.
The only reliable measure of strengths and weaknesses
is international comparison.
→ Enabling information systems for international
referencing (EU OMC indicators and benchmark, OECD
„Education at a glance” indicators, participation in
international student achievement surveys, connecting
internal and international assessment systems, etc.)
→ Ensuring the contextual relevance of international
comparative information by further analysis and
comparative research (See: European educational
performance patterns)

13. Determining quality and equity related
information needs
Connecting the results of meta-evaluation with the instruments
of quality evaluation
The characteristics of effective schools
General effectiveness enhancing factors
· Achievement orientation and high expectations
· Educational leadership
· Consensus and cohesion among staff
· Curriculum quality and opportunity to learn
· School climate
· Evaluative potential
· Parental involvement
· Classroom climate
· Effective learning time
· Structured instruction
· Independent learning
· Differentiation
· Reinforcement and feedback
Scheerens et alia. 2003.

14. Interpretation:
research and communication
• Research for deeper understanding: e.g. what makes
heterogeneous schools more capable to compensate
for disadvantages?
• Typical problem: research substituting information
production versus research for the interpretation of
information produced on a regular basis.
• Communication: transforming evidence to clear policy
messages. → „Simplexity” (Ora Ito) E.g. „Children in
schools with heterogeneous intake develop better.”

15. A few possible traps
• The trap of measurement: what we measure becomes a
problem, what we don’t measure remains invisible and
ignored
• „Economism”: considering the indicator equal with the
indicated fact, policies aiming at „improving indicators”
instead of solving problems
• Growing complexity → the fear of loosing control →
escaping from complexity by maintaining (or returning to)
centralization created simplicity
• Striving for the „optimum” instead of striving for the
„possible” (ignoring power, influence, prejudice, whim, etc.
by focusing only on evidence)
• Evidence as legitimacy substitute

16. Finally…
What determines
the quality of the
policy discourse:
the technology and
references of
policy-making
What has changed in Hungary between 2010 May and July?
Politics, nothing else.