Getting Public Health Research into Policy: Science, Art or Dance

www.hertsdirect.org
Research to Policy:
Science, Art or Dance?
Prof Jim McManus, OCDS, CPsychol, CSci, FFPH, FRSB, FRSPH, AFBPsS
Director of Public Health, Hertfordshire County Council
Jim.mcmanus@hertfordshire.gov.uk
July 14th
2016,
UK Public Health Research Centres of Excellence Conference

www.hertfordshire.gov.uk
What I want to cover
• A slightly tongue-in-cheek journey
• Poking fun at a relationship which doesn’t quite work,
because I care so much that it should
• Seen policy making in various places over 25 years
• Practical and real world view of research interface with
policy
• Things that work and things that don’t work
• What not to do
• Failings of Policy Makers

Longer Version
• A range of case studies including some from
criminology
• Some further tools for some of the steps

A Thesis...
• Public health is
inherently political
• Because it affects
populations
• It inevitably impacts
on or is impacted on
by Policy
• Need to think much
more about these
issues and claims

An average week for a chief officer/senior
civil servant
• 200 - 500 or more emails a
day
• Pile of FOIs, Surveys, Reports,
MP letters, Briefings
• Politicians confused about
media reporting
• Policymakers dissing experts
and trying to assert integrity of
expertise
• NGOs, Advocates,
Government all want a piece
• Lots of people “with solutions”
wanting time
• 63 meetings, many of them triple
booked
• Series of conflicting deadlines
• Reducing budgets
• Things come up unpredictably
which need “solving”
• Wanting to do the right thing for
the population
• Trying to get inside the
Politicians and Partners heads

Strategic Challenges
1. Delivering core public health must dos for less
2. Growing demand and need
3. The challenges of preventing cost and adverse
outcome (Falls going up in some areas)
4. Levering whole system redesign and prevention using
everyone’s resources to save money and improve
outcomes
5. Doing stuff which works
6. Massive and confusing literature on implementing
research

The 3 or 4E’s Duty or Value for Money
Duty
• Economy -value for money and delivering the required
service on budget, on time and within other resource
constraints
• Efficiency- getting an acceptable return on the money
and resources invested in a service.
• Effectiveness - extent to which the organisation
delivers what it is intended to deliver.
• Equity – not always in there even with legal
frameworks

What Policymakers think of researchers
Q. How many scientists does it
take to reach a firm conclusion?
A.There are too many
parameters to be definitive here

What Politicians Think...
• What is needed?
• What is right?
• What is doable?
• What is politically and financially doable?
• Who’s in my way? Who’s with me?
• What do my voters want?
• What do experts think?
• What do I now think?............Conclusion

The “knowing” and “interpretive” world
of policymakers
• The legacy (good and bad) of New Public
Management...measure everything, quantify
• Money, existing policy and law, influence
• The desire to do the right thing or the desire to please
political masters. Why are they in the job?
• Who’s got a view and how to I balance them?
• There’s no money
• What is the space for the possible when you take all this
into account

The Epistemology of the Policy Process
What they teach you at government school –
the science
Source http://www.thisnation.com/textbook/processes-p

The Policy Process
What usually tends to happen – the art of influence
1001 issues and influencers, many competing
Not everyone
happy
Policy
Proposal
Policy
Proposalrefinery
Refinement
Compromise
Partial
Implementation or
Effectiveness
Repeat loop eternally

More like a dance than art or science
• Requires instinct and “feel” for where you are in
the dance and why you’re doing it
• Requires good evidence and knowing when to
use it
• Requires relationship “heart speaks to heart”
• Requires emotional engagement
• Partnership with the other dancer(s)
• Adapting steps to where you are

The Policy Process Dance: Art & Science in relationship
The Policy Networks and Influences Approach
Source:
http://www.lancaster.ac.uk/fass/projects/hv
p/newsletters/5findings.htm

The Dance Floor - Policy Process in 5 Dimensions
A model for dynamic tension
Policy
Process
Quadrilateral
“Evidence”
The
Desirable,
The
Achievable
Motivations
Influences
Multiple layers:
1. Advocates
2. Politician
3.Officers
4. Scientists
5. Users
Across Time too

www.hertsdirect.org
Onto the dance steps....If that’s how
it happens, what works for the
Policy/Research relationship?
I have a personal proposal...

Περιχώρησις - Research, Policy and
Gregory Nazianzen
• Idea of περιχώρησις perikhōrēsis
• Dancing round. Mutual Intimacy.
• Interprenetration of understanding
• Dynamic hermeneutic unity
but not identity
• Psychology, Emotion, Mutuality
• Perikhoresis as a model for research
into policy

What to do – dance etiquette
• Build a Relationship!
• Research on who you need to talk to
• Focus on what you want to say and why they
should listen
• Hone
• The 60 Second Pitch to the right person made
personally
• And the usual newsletters etc

Dance Steps – 7/8 C’s of Consulting
1st
for me – What’s your Capital to
offer?
1.Client – who, what do they want?
2.Clarify – what is the dance?
3.Create – work up the doable
4.Change - what is to be done
5.Confirm – the what and the doing
6.Continue – keep doing it
7.Close – thanks for the dance, happy?
Read one review of this here
http://www.sweeneycomms.com/The-Seven-Cs-of-Consulting-g.asp

What not to do..wrongfooting
• Send a bulletin and expect anyone to read it
• Write in scientific terms for policymakers
• Set out what you’ve done without setting any
context
• Send a 99 page report with eight appendices
and no executive summary
• Send anything before you’ve worked out who to
talk to

Research on Diet
Is your research clear and conclusive enough to
give definitive guidance to people?
If yes – be clear and don’t be vague
If no – then shut up and continue
researching until it is clear enough
A moral responsibility on scientists NOT to
overstate their case and generalizability

Family Nurse Partnership – applying
the evaluation to public health decision
making
• Evaluation report
says doesn’t work not
cost effective
• FNP says it does
other valuable things
• Where do these
contradictory views fit
in policymaker
decisions?
• Issues raised:
– Appraisal of research
– By products of
interventions and
roundedness of
evaluations
– Applicability

Research on effects of e-cigarettes
• Most of it unappliable because of design flaws
or even repeating existing badly done research
• In vitro studies which don’t reflect real life
• Speculation and going beyond the research
doesn’t help

You could interchange these headings..
Policy is...
• Messy and Spiral
• Contested Space
• “Oh God, someone’s had
an idea”
• “We must do something”
• “Any evidence”
• “I want answers”
• “What will it do”
• Compromise, Influence
and Change
Research is..
• A trade off between what
you want to know/do and
resources to do it
• A methodological trade off
• Hermeneutically situated
• Contested

Knowing your partner...feasibility
Option A Option B Option C
Effectiveness Very Positive Positive No impact
Flexibility Very Positive Positive Positive
Sustainability Positive Positive Negative
Political Feasibility High Medium Low
Administrative
Feasibility
High Medium Low
Time Short Medium Long
Cost High Medium Low
Suggested criteria example

Communication - Writing style as an
example
Academic
• Heavily referenced
• Third person
• Passive voice
• Tentative conclusions
Local Govt
• Active voice
• Summaries
• Recommendations
• Costings
• Policy options
• Impact measurements
• Feasibility
• risk

Two partners in the same dance – i) the
research itself and ii) the organisation and
its agility

Soft skills for the dance....making the 8
Cs live
• Contested space – competing demands
• Learning and mutual understanding
• influence, nuance
• Understand what constitutes evidence
• Understand each side’s concerns
• Understand the pressure of being a policymaker
and the often limited analytical tools and
apparatus available
• Craft a working relationship in that space

Formation for the Dance not Training
for the Science
• Policy makers are formed – psychological,
political, emotional, conceptual, skills and tools
• Researchers need to be formed not trained in
how to work with one another
• A conscious relationship and working with one
another
• Too little interpenetration of the two
• Policy Networks

Thank you
• Jim.mcmanus@hertfordshire.gov.uk
• Public Health Pages
• www.hertsdirect.org/healthinherts
• JSNA
• http://jsna.hertslis.org/

www.hertsdirect.org
Slides for a longer session

Tools for the road....
• Contested space – competing demands
• Learning and mutual understanding
• Soft skills – influence, nuance
• Understand what constitutes evidence
• Understand each side’s concerns
• Understand the pressure of being a policymaker
and the often limited analytical tools and
apparatus available
• Craft a working relationship in that space

Some problems
• Massive literature on applying research in
practice
• Lots of problems in the literature
– Sometimes conceives receivers of research
as in deficit of knowledge mode
– Focuses often on one side – the research
quality or the organisation – not synergistic
enough

Thesis
• Applying research needs a partnership between
researchers and policymakers from the
inception if possible
• Two sides of a coin : the quality of the research
and the agility and leadership of the
organisation
• Develop a roadmap jointly

Context
• Cuts to public services
• Increasing demand on NHS services
• Financial unsustainability of current system
• We need research which can help us keep
people independent and healthy, and reduce
need for public services

Nationally funded research
• http://www.nihr.ac.uk/funding/public-health-
research-programme.htm
• http://www.nets.nihr.ac.uk/programmes/phr
• http://www.nihr.ac.uk/funding/school-for-public-
health-research.htm
• Great these are funded but we need more
emphasis on translatable research and
interventions, and evaluating existing ones

Case Studies from Criminology
• Very policy directed field
• Political and worldview issues in researchers
and politicians
• Negotiating a sometimes difficult space
• Government commissions a lot of research to
inform policy
• Some ongoing controversies
• Analogies for health work...

Framing the debate can put your
partner off…
Knowing your
partner –
Kenneth Clarke
or Michael Gove
would like this.
How would it go
down with
Michael Howard
or Theresa
May? Hulley et al, Brit J Crim, 2016

Why are you writing? To Change
Policy or just lay bare a problem?
Feeling
good about
what
you’ve
written but
what have
you
changed?
Raymen, Brit J Crim, 2016

Actors in the System
• Actors in
the system
can derail
the
implementat
ion, even
when trying
to do “the
right thing”

Study of Public Health Policy Making
Wilmot et
al, Journal
of Public
Health
(2016)

For real world relevance:
research cannot be static when life is dynamic
Limitation?
Or actually a
strength that it
addresses
dynamic real
world decision
making?
Wilmot et al, Journal of Public Health (2016)

A good example of policy revelant
work
• Frames a
policy problem
and how to
respond to it
relatively
simply
Walker et al, JPH, 2016

The Problem
• Outcomes Frameworks
• Connecting evidence with outcomes
• Connecting evidence with interventions
• Interests of policymakers and interests of
researchers

So how do you go about applying
research into public health practice?
• Lots of literature on this
• Some very complex and complicated models
• A Proposal from me
1. Treat it like a change management process
2. Think about two major sides to a coin
3. Project and process manage it
4. Fidelity and sensitivity test

Change Management
• Lessons from ESRC evaluations of large scale
IT programmes
• Lessons from NHS and Social Care research on
applying research

Levels and Phasing
• Which level is the intervention?
– Societal
– Environmental?
– Biological?
– Intrapersonal?
• Which phase of time?
– 1 year, 2, 3?

Return on Investment as part of the
argument
• Variable response from commissioners to this
• Too much use of poorly developed ROI models in
evidence
• ROI is not always the answer and sometimes doesn’t
even help
• Most ROI models aren’t real world enough
• Depends on how robust it is
• Who cashes the returned cheque? If agency A invests
but Agency Y gets the return, where is the incentive?

Return on Investment Tools
• Helps if the evidence is also very clear
• Helps if there is a timescale to the return and
clarity on who gets it
• The policy/commissioning/research partnership
is crucial to making ROI useful

What is evidence-based public health?
 Many definitions
◦ From Evidence-Based Medicine to Evidence-Based Public Health
 Evidence-based public health is defined as the development, implementation, and
evaluation of effective programs and policies in public health through application of
principles of scientific reasoning, including systematic uses of data and information
systems, and appropriate use of behavioral science theory and program planning
models. (Brownson, Ross C., Elizabeth A. Baker, Terry L. Leet, and Kathleen N.
Gillespie, Editors. Evidence-Based Public Health. New York: Oxford University
Press, 2003.)
◦ E-Roadmap to Public Health Practice Concepts (New Hampshire
Institute for Health Policy and Practice)
 Developing, implementing, and evaluating public health programs or public health
policies (in public health terms an "intervention") that have 1) data demonstrating
their effectiveness and 2) a grounding in a health behavior theory or ecological
model of health.

What is evidence-based public health?
• Many definitions
– “The process of integrating science-based
interventions with community preferences to
improve the health of populations.” (Kohatsu
ND, Robinson JG, Torner JC. Evidence-
based public health: An evolving concept.
Am J Prev Med. 2004 Dec;27(5):417-21.)

Steps for EBPH
• Develop an initial statement of the issue
• Gather data to quantify it
• Use the research literature to determine what is
already known
• Develop program or policy options
• Create an implementation plan
• Evaluate the program or policy plan
O'Neall, M. A., & Brownson, R. C. (2005). Teaching evidence-based public health to public health
practitioners. Annals of Epidemiology, 15(7), 540-544.

More on the 2 sides of the dance:
Partner 1: the Research itself
• Internal and external validity
• Applicability
• Clarity of methods and outcomes
• Translatability
• Intervention readiness
• Engagement with policymakers

Dance partner 2: the organization and
its agility

Example: Behavioural Sciences
• Review of Research
• Synthesis into policy paper
• Applying resource
• Articulating work and programme plan
• Delivery and championing
• Integration into strategy
– This is very high level and much more
involved in this

Develop an initial statement of the issue
Use PICO (from EBM) to begin define question
◦ P: Population
◦ I: Intervention
◦ C: Comparison
◦ O: Outcome
Use a logic model to begin your strategic planning
◦ Inputs, activities, outputs, results (short/long term)
◦ W.K. Kellogg Foundation Logic Model Development Guide
 http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf
◦ Logic model tutorials (CDC)
 http://apps.nccd.cdc.gov/dashoet/logic_model_1/menu.html
 http://apps.nccd.cdc.gov/dashoet/logic_model_2/index.html

Mapping the Policy Process
• General Context issues – domestic and
international.
• Specific Policy Issues (i.e. the policy cycle)
• Who are the Stakeholders? (Stakeholder analysis)
– Arena: government, parliament, civil society,
judiciary, private sector.
– Level: local, national, international
• Process matrix + political influence ratings
• What is their Interest and Influence?
[Sources: M. Grindle / J. Court ]

Ontario Drug Programme (Khan et al,2014)
http://dx.doi.org/10.1016/j.healthpol.2014.0
6.007
• Policymakers have cited barriers to using evidence,
including lack of research relevance and timeliness.
• .Although reports often demonstrate an increase in
research relevance, rarely do they provide concrete
methods of enhancing research timeliness
• . Additionally, the impact of researcher–policymaker
collaborations is not well-discussed.

Be clear about how feasible it is to do
this
Option A Option B Option C
Effectiveness Very Positive Positive No impact
Flexibility Very Positive Positive Positive
Sustainability Positive Positive Negative
Political Feasibility High Medium Low
Administrative
Feasibility
High Medium Low
Time Short Medium Long
Cost High Medium Low
Suggested criteria example

The problems
• Early and avoidable disability and disease
• Early avoidable death
• Smoking prevalence uneven
• Obesity rising in adults, flattening in children in
most places, rising in some
• Alcohol related disease on the increase
• Preventable cost to public health
• Multiple needs, worklessness
The Tartan Rug and local profiles give much more info on this
david.conrad@hertfordshire.gov.uk

Getting Public Health Research into Policy: Science, Art or Dance

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Getting Public Health Research into Policy: Science, Art or Dance

Similar to Getting Public Health Research into Policy: Science, Art or Dance (20)

More from Professor Jim McManus AFBPsS,FFPH,CSci, FRSB, CPsychol

More from Professor Jim McManus AFBPsS,FFPH,CSci, FRSB, CPsychol (20)

Recently uploaded

Recently uploaded (20)

Getting Public Health Research into Policy: Science, Art or Dance

Editor's Notes