Transferring knowledge into policy and the role of Wikiprogress

Kate Scrivens
Wikiprogress Project Manager
OECD Statistics Directorate,
Monitoring Progress & Well-being
Division

KNOWLEDGE & POLICY: OVERVIEW
 Introduction
 Theories of knowledge use in policy
 Case study 1: Programme for International Student
Assessment (PISA)
 Case study 2: Climate change measurement
 The policy impact of well-being/QoL measurement
 The role of Wikiprogress

INTRO THEORY PISA CLIMATE WELL-BEING WIKIPROGRESS
INTRODUCTION
 The OECD
 The OECD’s work on well-being measurement
 Wikiprogress
 What do we mean by “knowledge”?
 Well-being and Quality-of-Life measures as policy
knowledge

WHAT IS THE OECD?
 The Organisation for Economic Cooperation and
Development:
 Currently 34 member countries
 Historical origins: administering the Marshall Plan,
fostering economic growth
 Today’s mission: “Better Policies for Better Lives”
 A think-and-do tank, semi-autonomous, functioning
through peer review and consensus-building

THE OECD AND WELL-BEING
 2004: the 1st OECD World Forum on Statistics, Knowledge
and Policy
 The well-being and ‘Beyond GDP’ movement: diverse and
international
 2009: the Stiglitz Report
 2011: the launch of the Better Life Initiative
 BLI, How’s Life?, Subjective Wellbeing Guidelines, etc.
 High-Level Expert Group on Well-being (‘Stiglitz 2’), 5th World
Forum to be held in 2015

WIKIPROGRESS
 An online platform for sharing knowledge and data
 A community portal for anyone interested in well-being/
progress measurement and policy
 Hosted by OECD but open-source, independent
 Purpose:
 Provide resources and information for researchers,
practitioners and policy-makers
 Raise public awareness about well-being/progress issues

WHAT IS ‘KNOWLEDGE’?
 Knowledge = information + understanding
 Where does our information come from?
 Scientific research
 Policy research
 Official statistics (surveys, administrative data, etc.)
 Other data (Open government data, ‘big’ data, etc.)
 Media
 Experience
 Other people (friends, experts, advisors, conference
presenters, etc.)
 We cannot assume that knowledge and information are
objective and free of bias

THEORIES OF KNOWLEDGE USE IN POLICY
 Models of research use in policy
 Types of policy outcomes resulting from knowledge
use
 Political economy theories and the role of knowledge:
 Ideas
 Institutions
 Interests
 Communicating for policy: complexity vs accessibility

MODELS OF RESEARCH USE IN POLICY
 The policy process is rarely linear and clear-cut
 6 models of research use (Weiss, 1979, “The many
meanings of research utilisation”)
 Problem-solving model
 Knowledge-driven model
 Interactive model
 Political model
 Tactical model
 Enlightenment model

PROBLEM-SOLVING MODEL
 Based on the idea that decisions are made in a linear
series of steps
 A problem is defined
 Different courses of action are identified
 Pros and cons of each option are clearly set out
 The option offering the best solution is selected
 The policy is implemented
 The outcome is evaluated
 Represents most people’s ideal of evidence-based
policy-making
 BUT ignores the role of politics in the policy process

KNOWLEDGE-DRIVEN MODEL
 Sees academic research as the driving force of policy
development
 While academics may assume this to be norm, it is rare
in real-life policy
 Academic knowledge in the social sciences is usually
not undisputed or authoritative enough to lead directly
to implementation

INTERACTIVE MODEL
 Recognises the plurality of policy actors
 Input into the policy process can come from many
different sources (both formal and informal
 Practitioners, aides, researchers, lobbyists, journalists,
friends, etc.
 Difficult to trace the role of specific kinds of
‘knowledge’ in influencing policy
 Each actor provides a piece of the puzzle, but it is rare
that any one source alone can provide an answer

POLITICAL MODEL
 Political interests around an issue predetermine the
position that decision makers take
 Discussion around an issue has gone on for so long
that opinions are hardened and new evidence is
unlikely to change these positions
 Information is used selectively as ammunition to
support previously held positions

TACTICAL MODEL
 Information used for purposes unrelated to the actual
content it presents
 Examples
 A politician using statistics in a speech only to give the
impression of evidence-based thinking
 A mayor using a report as an excuse to delay city
planning reform (“we need to study the issue more”)

ENLIGHTENMENT MODEL
 Research and data are diffused through various
channels
 Conference presentations, news reports, discussions with colleagues,
indicator reports, scientific evidence etc.
 Generalisations emerge which provide decision makers
with the means of framing issues in a certain way
 Makes policy makers more sensitive to emerging
issues and helps to define problems
Weiss argued that this was the most common way that
social science research contributes to policy making

ROLE OF INDICATORS
 The BRAINPOoL project (following on from POINT), adapted
these general models to the role of indicators.
 Instrumental use, where indicators are seen as objective
information tools to improve policy making, solving problems
and consciously influencing decisions
 Conceptual use, which sees the prime value of the indicator
to be more intangible, influencing how policy makers define a
problem
 Political use, which consists of 3 sub-categories
 Strategic use: used to bolster a position/decision that has already been
taken
 Tactical use: where decisions are postponed or avoided with the excuse
that data is being awaited
 Symbolic use: where indicators are used only to convey a message or
present an image

TYPES OF POLICY OUTCOMES FROM
KNOWLEDGE USE
 Content change
 Change in the content of policy and/or resources
allocated
 Agenda setting change
 Change in policymakers’ priorities and where attention
is drawn to new issues that were previously ignored
 Framing shift
 Changes the way that policymakers understand a
problem or the possible responses to it

TYPES OF POLICY OUTCOMES (CONTD.)
 Procedural change
 Changes how policy itself is made by procedural/ institutional
change that leads to new actors or new evidence being part of
the process of decision-making
 Behavioural change
 Change in the way in which policy is implemented
 From Sumner et al. 2009, “Making science of influencing:
Assessing the impact of development research”

POLITICAL ECONOMY THEORIES & KNOWLEDGE
 Mainstream economic theory provides a framework of
understanding based on idealised models of human
and market behaviour
 Theories of political economy aim to take into account
the ways policies are developed and implemented in
the real world
 Political economy theory tends to cover three broad
areas (the 3 ‘I’s):
 Ideas
 Institutions
 Interests

IDEAS
 Ideas, in this context, refer to our implicit and explicit
assumptions about how the world works
 No matter how objective and scientific we think we are being,
we tend to operate within certain cognitive and normative
frameworks
 Ideas are powerful – and they are most powerful when we are
not even aware of their influence: many things we take to be
“fact” are socially constructed
 Dominant ideas tend to change gradually over time, but they
can also change quickly after a shock
 Ideas (norms, theories, shared understandings, worldviews,
paradigms, ideologies) both constrain and enable collective
action

INTERESTS
 In all societies there are different groups who have different
interests in the outcomes of policies (interest groups)
 Groups that have more resources, more power and are
better-organised tend to have the most influence on policy
outcomes
 This is especially the case where the costs of a policy are
concentrated within one group, and the benefits are
distributed across society (or vice versa)
 However, politicians need votes as well as support from
elites: where widespread public opinion can be mobilised,
this can provide the political will to override powerful lobbies
 For some, interests and ideas are two sides of the same coin

INSTITUTIONS
 Institutions are the rules and laws that govern society, and
the bodies and processes which implement them
 Institutions shape the way in which decisions are made (e.g.
consensus, majority vote, expert-driven, evidence-based,
negotiation) and policy action is implemented
 Institutions matter a great deal for determining the impact of
knowledge on policy
 Institutional change tends to be slow

COMMUNICATING STATISTICS FOR POLICY
 Statistical products generally have three types of audience
1. Statisticians/researchers
2. Policy makers/analysts
3. Public (media, civil society, non-experts
 Each of these audiences requires a different amount and
complexity of data
Public
Policy
Tech
Public
Policy
Technical
Size of
audience
Amount/
complexity
of data

KEY POINTS FROM THE THEORY
 There is no one way in which knowledge can be
transferred into policy
 Multiple actors from different sectors of society can play
a role in the policy process
 The impact of knowledge on policy can take many forms,
and is often diffuse, subtle, accumulative and hard to
trace
 Scientific knowledge, no matter how convincing and
clear-cut it is, will be filtered through the dominant ideas,
interests and institutions of a society, which will shape its
impact on policy
 The communication of knowledge for policy needs to be
tailored to different purposes and audiences

WHAT HELPS INDICATORS IMPACT POLICY?
 According to the BRAINPOoL project, four “success” factors
can be identified for policy indicators
 Salience for decision makers
 Indicators are successful when they can demonstrate real relevance for
policy or strategy
 Salience for a broader audience
 Producing a simple, attractive message that links to a meaningful concept
while avoiding certain ‘taboo’ words and concepts is key to impacting a
broad audience
 Credibility
 Data quality and the appearance of neutrality are important
 Relationships
 Relationship-building with the intended audience to strengthen the
legitimacy of indicators is key

CASE STUDY 1: PISA
 The Programme for International Student Assessment
(PISA) – description and objectives
 Implementation of PISA
 Policy impact
 Lessons from PISA on how knowledge gets transferred
into policy

PISA: DESCRIPTION AND OBJECTIVES
 The Programme for International Student Assessment
(PISA) is an international survey which aims to
evaluate education systems worldwide to inform
national education policy
 More than 70 economies have participated in the
assessment to date
 The survey is held every three years, with the main
focus rotating between three core subjects: literacy,
mathematics and science
 The most recently published results are from the
assessment in 2012, in which over half a million
students took part

IMPLEMENTATION OF PISA
 PISA was created in 1997, with the first round of
assessments taking place in 2000
 Each round of PISA operates on a 5-year cycle (3 years’
preparation, 2 years’ reporting)
 The development and implementation of PISA is based
on collaboration and consensus with a wide range of
experts and practitioners
 The institutional structure of PISA is geared towards
high levels of cooperation and exchange between the
OECD and national experts and policy makers in
participating countries

POLICY IMPACT OF PISA
 Large-scale reform has been implemented in several
countries as a direct result of PISA “shock”
 E.g. Germany, Switzerland, Denmark, Japan
 The influence of PISA has increased over time and it is
now widely recognised as the most important
international evaluation of education systems
 It is a good example of instrumental use of indicators
that has led to a wide range of policy outcomes
(content, procedural, agenda-setting, framing,
behavioural change)

LESSONS FROM THE PISA EXPERIENCE
 Why has PISA been so immensely influential?
 It meets all 4 BRAINPOoL criteria for successful
indicators (saliency for policy and public, credibility and
relationships)
 It covers a well-defined problem (education) whose
importance is widely-agreed across different groups (few
opposing interests)
 It represents the dominant idea that education is
important for economic productivity
 The institutions and tools to implement education reform
are well-established

CASE STUDY 2: CLIMATE CHANGE
 The evolution of knowledge on climate change
 The policy impact of climate change evidence
 Lessons from the example of climate change

KNOWLEDGE ON CLIMATE CHANGE
 Scientific concern over the global warming and the role of
human activity began in the 1950s
 By the 1990s, a scientific consensus had emerged around
the need to act
 1988: the Intergovernmental Panel on Climate Change
(IPCC) was formed
 1992: United Nations Framework Convention on Climate
Change
 1997 (enforced 2005): Kyoto Protocol (first legally binding
agreement)

KNOWLEDGE ON CLIMATE CHANGE (CONTD.)
 By 2000s, volume of evidence on the need to take
policy action to combat climate change is
overwhelming
 2007: Stern Report on Economics of Climate Change
(estimated 1% of world GDP needed to be invested)
 Nov 2014: Latest IPCC Report released:
“Without additional mitigation efforts beyond those in place today, and
even with adaptation, warming by the end of the 21st century will lead to
high to very high risk of severe, widespread, and irreversible impacts
globally”

POLICY IMPACT OF CLIMATE SCIENCE
 While the scientific knowledge is unanimous, policy
action has lagged behind
 Countries’ commitments have tended not to be
matched by action
 The Kyoto Protocol took years of negotiations to be
enforced, and the US has refused to ratify the treaty
 Implementation of policies to reduce carbon emissions
are extremely uneven across countries (tending to be
stronger in EU than non-EU countries)

LESSONS FROM CLIMATE CHANGE
 Climate change is a complex issue, which touches
upon a wide range of economic, social, technological
and institutional challenges
 While the message and evidence is clear, the political
and institutional barriers to action are significant and
numerous:
 Strongly entrenched opposing interests promoting a
contradictory narrative
 National institutions are not well-equipped to deal with cross-cutting
issues with high costs of action and diffuse, delayed
outcomes
 Lack of political consensus around the best way to act on
climate change (few established policy tools and levers)

LESSONS FROM CLIMATE CHANGE (CONTD.)
 Public knowledge of climate change is a powerful
determinant of climate change policy adaption
 Countries in which the public is aware of the causes of
climate change are significantly more likely to adopt
climate change mitigation policies

WELL-BEING AND QOL MEASUREMENT
 The OECD and well-being measurement
 The policy impact of OECD well-being work
 Lessons learned
 Towards the development of tools for using well-being
measures in policy

GDP “measures neither wit
nor our courage, neither our
wisdom nor our learning,
neither our compassion nor
our devotion to our country, it
measures everything in short,
except that which makes life
worthwhile”
GDP is a flawed measure of people’s actual well-being, and
therefore an incomplete measure of a nation’s true progress

THE FIRST SOCIAL INDICATORS MOVEMENT
 The current well-being/progress/QoL/Beyond GDP
movement is not new, but the political momentum is
 In the 1960s and 1970s, a movement to develop social
indicators emerged in North America and Europe
 NASA project in the 1960s
 1970: OECD Social Indicators programme established
(measuring health, education, employment and quality of
working life, time and leisure, command over goods & services,
physical environment, social environment, and personal safety)
 1974: launch of Social Indicators Research
 However, by the mid 1980s, most social indicator work
disbanded or significantly reduced due to lack of policy
support

THE OECD WORLD FORUMS
 In 2004, the OECD held the 1st World Forum on
Statistics, Knowledge and Policy in Palermo.
 Purpose was to bring together a wide range of societal
actors – not just statisticians – to talk about statistics
 Rather than being a technical conference, it looked at
the fundamental role of statistics to create knowledge
for policy, and to underpin democracy: “we measure
what we value”
 It recognised that statistics are not politically neutral,
and that deciding the best way to measure ‘progress’
requires an inclusive approach

WHY ARE STATISTICS ‘POLITICAL’?
 Counting requires decisions about categorising, about
what (or whom) to include and exclude
 Measuring any phenomenon implicitly creates norms
about what is too little, too much, or just right
 Numbers can be ambiguous, and leave room for political
struggles over interpretation
 Numbers are used to tell stories, such as stories of
decline
 Numbers create the impression that very complex
phenomena are simple, countable and precisely defined
 Numbers can create political communities out of people
who share some trait that has been counted
 Numbers, by seeming to be so precise, help bolster the
authority of those who count

THE ISTANBUL DECLARATION
 The 2nd World Forum, held in Istanbul in 2007 led to the
Istanbul Declaration, where signatories pledged to:
 Encourage communities to consider for themselves what
“progress” means in the 21st century
 Share best practices on the measurement of societal progress
and increase the awareness of the need to do so using sound
and reliable methodologies
 Stimulate international debate, based on solid statistical data and
indicators, on both global issues of societal progress and
comparisons of such progress
 Produce a broader, shared, public understanding of changing
conditions
 Advocate appropriate investment in building statistical capacity,
especially in developing countries

FROM DISCUSSION TO MEASUREMENT
 OECD continues to hold the World Forums every 2-3
years (5th World Forum to be held in Mexico in 2015)
 However, there has been a shift in OECD work from
agenda-setting/ framing to developing policy measures
 Stiglitz report in 2009 gave impetus and laid out the
roadmap for methodological work
 Greater emphasis on household perspective and patterns of
distribution (inequalities)
 Recognition that well-being is multidimensional
 Need for both subjective and objective measures
 Focus on both current well-being and sustainability

OECD WELL-BEING FRAMEWORK

OECD WORK ON WELL-BEING
Plus: country reports, working papers,
Income Distribution Database,
Inclusive Growth Index, Big Data
analysis, Handbooks and Guidelines,
High Level Group, Forums

POLICY IMPACT OF OECD WELL-BEING WORK
 There is now a widespread consensus amongst
statisticians, policy makers, practitioners and
researchers that policy needs to go ‘beyond GDP’
 Many national leaders have made public declarations
of the importance of measuring well-being (UK,
Germany, United States)
 There are many national and sub-national initiatives to
develop broader indicators of progress (UK, Italy,
Israel) - based on public consultation
 Some national treasury offices have established well-being
frameworks to assess national expenditure (New
Zealand, Canada, Australia)

POLICY IMPACT OF OECD WELL-BEING WORK
 The OECD is only one of many actors working on these
issues, but widely seen as a leader (credible)
 Continued to advocate for well-being measures after
crisis
 While there is much conceptual and symbolic use of
well-being measures, examples of instrumental use is
rare
 To date, the policy impact of well-being measures has
been closest to the enlightenment and interactive
models - diffuse, with many actors, impacting primarily
through agenda-setting and framing e.g multiple
dimensions, importance of SWB

LESSONS LEARNED
 Lessons learned from the first social indicators
movement – why did it fail?
 Social issues became overshadowed by economic
concerns as the global economic situation worsened
 Ideological shift towards conservatism and reduced
government
 Limited usefulness of social indicators to policy makers
 Lack of unifying theoretical framework to compare with
economic theory
 Lack of common metric for aggregation comparable to
money in economics

LESSONS LEARNED
 Many of these factors apply now too (crisis, reduced
government, lack of unifying theory) – what is
different?
 Crisis has strengthened sense that traditional
measures were missing something important, and that
a disconnect exists between people’s experience and
aggregate economic measures
 This has supported the narrative (idea) that change is
needed

GDP VS LIFE SATISFACTION IN EGYPT BEFORE 2011
Traditional measures did not predict the Arab Spring,
whereas indicators of the share of people “thriving” in the
country plummeted

LESSONS LEARNED
 But what are the barriers to instrumental use?
 Ideas are changing but institutional change is needed
too
 Well-being and quality of life are complex and multi-dimensional
 Unlike education, there are no ministries of well-being
 Policy discussions on well-being/QoL tend to exist in
parallel with the economic policy process

Source: BRAINPOoL final report, 2014

LESSONS LEARNED
 GDP predominates policy discussions because
 The prevailing policy belief is that economic growth is
the key goal for policy
 It allows for the combination of multiple factors into a
single, powerful number that immediately
communicates its meaning
 It is embedded in a widely-accepted theoretical
framework, with well-established institutions and policy
tools to support policy making focused on the
maximisation of aggregate productivity

LESSONS LEARNED
 How do we promote the goal of well-being as the
ultimate objective of policy-making?
 Ideas are starting to change but this will be a slow
process
 We need to continue to communicate knowledge on the
importance of well-being, tailored to different audiences
(including the general public)
 We need to appeal to people’s interests, with a convincing
narrative for change (e.g. to those still focused on economic
growth, focus on the economic benefits of well-being policy)
 We need to reform the way economics is taught to take into
account new thinking on well-being and inequalities

LESSONS LEARNED
 We need new institutional structures that are better
geared towards the delivery of well-being policy
 Realising the potential of well-being data to design
better policies “requires change to policy-making
processes at both national and local level, incuding
breaking down silos between policy areas and
reforming the process for allocating budgets”
(Report of UK All-Parliamentary Group on Wellbeing Economics)
 We need new frameworks and tools for well-being
policy analysis

POLICY TOOLS USING WELL-BEING MEASURES
 Composite indices
 Inclusive growth index, ecological footprint, human
development index
 Accessibility of information over
comprehensiveness/complexity
 Need to be embedded in an analytical framework
 Multiple criteria analysis
 Designed to deal with decisions with multiple objectives – not
all of which monetisable or easy to aggregate
 Weighting and trade-offs are made explicit
 Designed to assess different options, usually includes
stakeholder deliberation

POLICY TOOLS USING WELL-BEING MEASURES
 Subjective well-being (SWB) measures
 There is an increasing recognition of the need to measure
and maximise SWB
 SWB can be understood in a number of ways (affect, life
satisfaction, eudomonia)
 There are a number of ways in which SWB measurement can
be used in policy (principally focused on life satisfaction)
 To inform policy design, monitoring and evaluation as the
dependent variable (e.g. work, health, city planning)
 To provide input into other forms of analysis such as cost-benefit
analysis
 OECD has played a major role in increasing the profile of
SWB measures in policy

QUESTIONS
 What do you think policy makers should be maximising
(e.g. SWB, capabilities, economic growth)?
 How would you persuade a government leader of the
importance of taking into account well-being
measures?
 How would you persuade the general public of the
importance of promoting policy for well-being?
 How do you think knowledge on well-being is used in
policy in your country?

WIKIPROGRESS
 Wikiprogress is an online space for sharing knowledge on
well-being and progress
 It serves 3 main functions
 A knowledge base providing resources for anyone interested in
‘progress’ (articles, reports, data, organisations)
 A community portal for researchers, practitioners and engaged
non-experts (events, blogs, discussions)
 Communicating to a general public on the importance of
measuring well-being for better policies
 We would like your help!
 Wikiprogress depends on volunteer contributions
 We would like your help in creating a new portal for young
people

THANK YOU!
Wikiprogress.org
Kate.Scrivens@oecd.org
@katescrivens

Transferring knowledge into policy and the role of Wikiprogress

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Transferring knowledge into policy and the role of Wikiprogress

Similar to Transferring knowledge into policy and the role of Wikiprogress (20)

More from Wikiprogress_slides

More from Wikiprogress_slides (20)

Recently uploaded

Recently uploaded (20)

Transferring knowledge into policy and the role of Wikiprogress