The Use of Evidence in Policy Making: Practice and Prospects

The Use of Evidence in Policy
Making: Practice and Prospects
Prof. Ian Sanderson,
Director of Research, Faculty of Business and Law,
Leeds Metropolitan University
The Use of Evidence in Policy Making?
Leeds Social Sciences Institute Seminar, University of Leeds
22 Oct 2010

Outline of presentation
 Evidence-based policy making:
Conceptual and empirical issues.
 Use of evidence in government: some
examples
 Prospects for evidence in policy
making’?

The Government line
“If government decisions are to be robust, they need to
be based on all relevant evidence. Science and
engineering are key elements of this evidence base.
The UK is respected internationally for ensuring that the
right evidence is acquired and brought to bear on
issues. But more needs to be done to ensure that
policy makers have access to the best scientific
evidence and advice and are therefore in a position
to make the best decisions about the challenges,
both national and international, facing the country.”
(Science and Engineering in Government: An Overview of the
Government’s Approach, Government Office for Science, Oct. 2009)

The ‘Nutt Affair’
 Observer, 1/11/09:
 “The essence of democracy is evidence-based argument,
reason and genuine deliberation. Of course there will be a
passionate clash of values and priorities, but if we cannot
accept the facts we descend into a shouting match between
rival prejudices.”
 “…British politics – and our culture – is increasingly being
disfigured by politicians bowing to prejudice…Britain is
losing its way, unmoored from its tradition of fair play, debate
and respect for facts. Nutt’s sacking was another milestone
in Britain’s progression from a great Enlightenment country
into a place where prejudice reigns.”
 On EBP:
 “Nobody rational could possibly want a government based
on any other type of policy-making.” (Prof. Colin Blakemore,
Observer, 1/11/09)

A recipe for controversy?
 Lack of conceptual clarity  ‘evidence’ / ‘based’ /
‘policy’
 Weak empirical foundation / lack of robust theory
 generalisation from limited empirical basis
 Strong ideological/normative basis
 deep-seated controversy re. rationality/technocracy

‘Enlightenment’ values

critique of expertise/technocracy

politics/power
 cf. evidence-based policy or policy-based evidence?
  debate tends to conflate empirical/normative  ‘is
v should’

What constitutes
‘evidence’?
 ‘Valid’/relevant knowledge?
 explicit  tacit
 scientific/technical  experiential
 ‘objective/disinterested’  ‘knowledge with an interest’

the domain of knowledge is not confined to the one demarcated by scientists…” (Laws and Hajer, 2006,
p.416)
 Whose knowledge?:
 academic researchers
 ‘independent researchers
 consultants
 government analysts
 policy officials
 professional/practitioners
 managers
 service users/customers
 ‘think tanks’
 interest groups
 Politicians
 Social scientific knowledge rarely provides the definitive word on
what it is best to do and cannot provide the definitive word on what
we should do.
Increasingimportanceof
tacit/experiential/
‘knowledgewithinterest’?

What does it mean for ‘policy’
to be ‘evidence-
based/informed’?
 What is ‘policy’?
 ideologies/paradigms
 problems/needs
 strategies/objectives/targets
 measures/legislation
 practices/guidelines
 What type of influence/impact?
 instrumental  conceptual
 implementation  understanding problem/’policy paradigm’
 What degree/level of impact?
 determination  ‘degree of influence’ (amongst many factors)
 What means of influence?
 specific research study  expert advice  tacit knowledge
 ‘product’  ‘process’ (dialogue/relationships/networks)

Complexity/Contingency
 The extent to which, and way in which
knowledge in various forms is used in ‘policy
making’ depends on context/ circumstances
in a complex set of processes
 Much of the work in government involving the
application of ‘analysis’ to ‘policy making’ is
hidden  ‘back-room work’
  beware generalisation from partial/limited
evidence

Use of evidence in government:
Scotland
 Well-developed ‘analytical infrastructure’
 c. 300 analysts / analytical units covering all policy areas
 2006 ‘Capability Review’  “good use of evidence”
 SNP Manifesto commitments 2007:
 Reduction in business rates
 Removal of tolls on Forth/Tay bridges
 Replacement of Council Tax with LIT
 Abolition of graduate endowment fee
 Replacement of student loans with grants
 Minimum class sizes in P1-P3
 Phasing out of prescription charges
 Additional police ‘on the beat’
 …and…Scottish independence!
 Based more on political considerations than
evidence?

Use of evidence in government:
Scotland
 ‘Space’ for policy development beyond manifesto
commitments 2008-09
 Strategic policy frameworks:
 Early years and early intervention
 Poverty and Income Inequality
 Health inequalities
 Smoking
 Drugs
 Alcohol
 Offending by Young People
 Antisocial behaviour
 Extensive use of evidence: analysis of problem; ‘what
works’; public and stakeholder views

Example: Towards a smoke-
free Scotland
 ‘Reducing Smoking and Tobacco-Related Harm’,
2003
 ‘A Breath of Fresh Air for Scotland: Tobacco Control
Action Plan’, 2004
 Ban on smoking in public places, 2006
 Scottish Ministerial Working Group on Tobacco
Control
 Smoking Prevention Working Group

academic experts/policy officials/practitioners/ASH

‘Towards a Future without Tobacco (2006)
 recommendations based on review of evidence
 public consultation 2006-07
 ‘Scotland’s Future is Smoke-Free: Smoking
Prevention Action Plan, 2008
 Health (Scotland) Bill, 2009

Example: Antisocial
Behaviour
 Promoting Positive Outcomes: Working Together to
Prevent Antisocial Behaviour in Scotland (
http://www.scotland.gov.uk/Resource/Doc/264302/0079222.pdf)
 “We have arrived at this Framework…through an
inclusive, consensual and evidence-led process” (p.2)
 Sources of evidence:
 Expert Advisory Group
 Review of published research and evaluation
 Analysis of ASBO breaches
 Interviews with LA Antisocial Behaviour Co-ordinators
 Interviews with senior police officers
 Focus groups with members of the public
 Interviews with young people
 So, is policy evidence-based?  It depends!

Is the Coalition Government
committed to EBP?
 2010 Spending Review framework:
 Is the activity essential to meet Government priorities?
 Does the Government need to fund this activity?
 Does the activity provide substantial economic value?
 Can the activity be targeted to those most in need?
 How can the activity be provided at lower cost?
 How can the activity be provided more effectively?...etc.
 Independent reviews:
 Browne Review of Higher Education Funding
 Wolf Review of 14-19 Vocational Education
 Tickell Review of Early Years Foundation Stage
 Munro Review of Child Protection
 Local Government Finance Review
 Review of Offender Learning….etc.

BUT…….
 Child Benefit
 “The Chancellor’s Monday morning bombshell on child benefits was
raw politics and bad policy. There was dismay amongst very senior
civil servants, who still believed that this government was going to
be different. No more rushed, confused announcements in time for
the news headlines.”

(Secret Diary of a Civil Servant, Observer, 10th
Oct. 2010)
 ‘Bonfire of the Quangos’
 And…remember cannabis:
 “Cannabis has become the football in a context between evidence
and passion.”

(Colin Blakemore, Observer, 4th
May 2008)
 ‘Policy-based evidence’?
 “…we will never produce the type of knowledge base that
healthy progressive societies need when the government
mantra of “evidence-based policy” is so easily translated into
“policy-based evidence” (Tombs and Whyte, 2003, p.20).

What should we wish for?
Government makes extensive
use of evidence to inform policy
making and should do more as
the key basis for improving
policy.
‘EBP is an ideological
concept - policy making is
a fundamentally political
process.’
Policy making is informed by various
form sof knowledge and we should
improve policy making through
knowledge-informed deliberative and
learning processes.
Government makes insufficient use
of evidence, often using it to justify
and legitimise political commitments,
and should make more/better use to
improve policy.
Is policy making evidence-based?
Optimist Pragmatist Critical
Shouldpolicymakingbeevidence-
based?
Rational-
technocratic
PragmatistRadical-sceptic
Evidence-Based Policy
Policy-Based Evidence
Policy Making as Politics
‘Intelligent policy
making’
Policy Making as
Craft/’Phronesis’
Policy Making as
Science/’Techne’

Policy making as ‘deliberation’
 “Rooted in the tradition of American pragmatism of John
Dewey…the interpretative approach to public policy has already
contributed to a more subtle understanding of policymaking and
politics” (Hajer and Wagenaar, 2003, p.6).
 “Grounded in a commitment to public enlightenment, a
deliberative orientation…promotes a form of practical
knowledge that brings technical findings together with the
political values and social assumptions to which they relate, as
well as the action-oriented narratives required for decision
making” (Fischer, 2009, p.7).
 “(T)he choice of policy instrument is not a technical problem that
can be safely delegated to experts. It raises institutional, social
and moral issues that must be clarified through a process of
public deliberation and resolved by political means.” (Majone,
1989, p.143).
 “Argumentation is the key process through which citizens and
policy makers arrive at moral judgements and policy choices”
(Majone, 1989, p.3).

The key ‘ingredients’ of
policy
Effectivenes
s
Feasibility
Desirability/
Acceptabilit
y
(Social)
scientific
evidenc
e
‘Organisational evidence’/
‘practitioner wisdom’
Political
values/
policy
advocacy
Knowledg
e with an
‘interest’
Based on Coles (2010)

Prospects for ‘evidence’ in
policy making?
 “If we enlarge the meaning of evidence, there is indeed
scope for bringing more intellectual edge to the analysis of
what we can learn from the past. But, equally important, if
we remember that evidence speaks with many voices –
and that our values drive facts and shape the conclusions
we draw from them – we will also conclude that any such
exercise will be no more, and should be no more, than
one contribution to the process of policy making.”

(Rudolph Klein, 2003)
 “There will be no single institutional means to accomplish
this subtle balancing act of technical expertise, citizen
participation, and executive, since circumstances differ by
topic, urgency, public understanding, time horizon
involved, and much else. Yet the general commitment to
the combination of scientific rigour, public consensus and
democratic accountability should be a guidepost for action
and institutional design.”
Prof. Jeffrey Sachs, Science and Technology Newsletter, 4th
Nov. 2009, www.publicservice.co.uk

John Dewey
 “…(I)n the degree in which an active conception of knowledge
prevails … (c)hange becomes significant of new possibilities
and ends to be attained; it becomes prophetic of a better future.
Change is associated with progress rather than with lapse and
fall. Since changes are going on anyway, the great thing is to
learn enough about them so that we be able to lay hold of them
and turn them in the direction of our desires.” (Dewey, 1957,
p.116)
 “…the necessity of a deliberate control of policies by the method
of intelligence, an intelligence which is not the faculty of intellect
honored in text-books and neglected elsewhere, but which is
the sum-total of impulses, habits, emotions, records, and
discoveries which forecast what is desirable and undesirable in
future possibilities, and which contrive ingeniously in behalf of
an imagined good” (1993, p.9)

The Use of Evidence in Policy Making: Practice and Prospects

The Use of Evidence in Policy Making: Practice and Prospects

More Related Content

Viewers also liked

Similar to The Use of Evidence in Policy Making: Practice and Prospects

Recently uploaded

The Use of Evidence in Policy Making: Practice and Prospects