This presentation was given by Rien Rouw of the Ministry of Education of Holland at the CERI Conference on Innovation, Governance and Reform in Education on 3 November 2014 during session 3.a: Knowledge-intensive Governance, Innovation and Change.
7. 1. Evidence based medicine is the conscientious, explicit, and judicious use
of current best evidence in making decisions about the care of individual
patients. The practice of evidence based medicine means integrating
individual clinical expertise with the best available external clinical
evidence from systematic research (Sackett et al)
2. Integration of experience, judgement and expertise with the best
available external evidence from systematic research (Davies, Cabinet
Office)
3. Decisional capital as a third pillar besides 7
human capital and social capital
(Fullan en Hargreaves)
8. *Experimentalism
*Trial and error as the source for improvement
*Building coalitions
*Various perspectives as the source for deep
understanding
9. *Effectiveness insight, Cooperation in
experimental designs
* between policymakers and researchers at the
Ministry of Education
* policymakers need to translate randomized
research methodology into the design of an
intervention
* to create mutually comparable intervention and
control groups
* for instance by distributing a grant at random
10. *‘Evidence for Education’: special fund
* 23 million euro spread through competition
* Only consortia of research institutes and schools
could apply
* Methodological demand: exclusively RCT’s
* 37 experiments
11. *Brokerage needed: either a third party,
capable / specialized reseachers or
practitioners
* Good experiments need input from the field,
scientists with expertise in the field, scientists
with statistical expertise
* Insight in explanatory mechanisms is crucial
* Systemic approach needed
* Political tension is not a good condition for
experimental research
12. *Netherlands Initiative for Education Research (NRO)
* To program coherently fundamental, policy-oriented and
practice-oriented research
* To bridge the gap between research and practice
*Match Making Meetings
*Only consortia can apply for practice-oriented research
* To make research easy accessable for practitioners
13. *Regional partnerships between universities,
universities of applied science, teacher training
institutes and schools
*Close cooperation between researchers and
teachers
*Combined with teacher training
*Development of reflective practitioners
*Teachers as researchers
15. * Continuous interaction: involve
policymakers in generating a report
* Direct connections: no intermediary
organisations
* Dialogue with various research ‘schools’ /
traditions:
evidence from various perspectives, both
quantitative and qualitative,
both causality and explanatory
mechanisms
* Multiple channels:
scientific reports, short
summaries, video, blogs,
workshops, seminars etc.
* Multilevel: the level of facts, of discourses
and of beliefs
* Brokerage as specific function:
- organized as a specific function or
profession, not necessarily in a
separate institution
- exercised by people of a certain kind:
researcher nor policymaker, a go-between
by heart
* Capacity: of policymakers to interpret
research, to judge results and to process
them into policy
* Systemic: involves practitioners,
researchers, policymakers and mediators
15
Editor's Notes
What does complexity mean for the process of decision making in education practice and policy?
Where does knowledge come from when you are in a pressure cooker: it’s in their heads. Just as a doctor.
We want to stimulate the use of experimental research designs in policymaking. For policymakers this implies more than commissioning a certain research methodology. It means that the design of an intervention must be adapted to the requirements of randomized research. This is possible for instance by distributing a grant at random, in order to create mutually comparable intervention and control groups. We have set up several experiments where policymakers and researchers are working closely together in the first steps of the implementation of policy: the neighbourhood school (an institute for multiproblem dropouts), a fund for individual training grants for teachers and the extension of learning time for children at risk in primary education.
1) The first designing principle is that researchers and policymakers should maintain continuous interaction, or “sustained interactivity”, as it is also called (Sutcliffe and Court, 2005) or “interactive brokerage” (by one of your colleagues, Tom Leney). If researchers want their research to be used, they must involve policymakers in generating their report. It is not enough to present a plan at the beginning and the results at the end of the research. Furthermore, you have to invest in long-term relationships, because most policies are changing gradually (Levin 2005).
2) Policymakers and researchers should not only continually interact, they should also connect directly with each other. No intermediary organizations such as development institutions must come in between.
3) Policymakers must organize a dialogue with various research ‘schools’ or traditions. ‘Polygamy’ as some researchers call it (Nutley et al. 2003). You need evidence from various perspectives to understand deeply the complex world of education, both quantitative and qualitative research, both research into causal relations and research into mechanisms explaining causality.
4) To spread the good news of results, researchers must use multiple channels to reach diverse audiences: scientific reports, short summaries, video, blogs, workshops, seminars etc.
5) Results need to be communicated on more than one level: the level of facts, of discourses and of beliefs. If research leads to profound changes in policy it is not enough to present the data. Researchers must also formulate a message on the level of beliefs.
6) The previous points clarify the fact that brokerage between science and policy needs to be organized as a specific function or profession (f.i. Canadian Health Services Research Foundation, 2003). It appears preferable not to organize brokerage in a separate institution, but as a function either within the ministry or within research institutes. Furthermore, brokers need to be people of a certain kind. A broker is not producing knowledge, he is not using knowledge, he is not disseminating knowledge. He could be processing knowledge, but his main objective is to make connections between research and policy. Brokers are liaisons or connectors, sometimes they are called ‘boundary workers’. He must be an expert in both science and policy processes, curious, flexible, modest, sharp on details and at the same time understanding the bigger picture, a mediator and communicator. A broker succeeds if no one notices that he is connecting people and worlds. You could compare him with the old-fashioned telephone-operator whom we all know from the movies. The broker must be invisible.
7) However, all these designing principles will not be effective without one precondition being fulfilled, to wit the capacity of policymakers. Policymakers must be capable of understanding and interpreting the methods and results of research. They must develop evidence awareness, just as researchers need to develop policy awareness, by the way.
8) My last designing principle holds especially when profound changes are proposed. In that case an intervention is needed on the level of the whole system of education practice, research and policy. For the promotion of evidence based policy and practice in the Netherlands we call it an eco-system, which consists of three angles: practice, research and policy. A systemic approach means that all subsystems are involved and that diverse streams of interaction improve.