The University of Sydney Page 1
Increasing the use of
research in policy
agencies: what
works?Presented by
Gabriel Moore
School of Public Health University of Sydney and the Sax Institute
Danielle Campbell
Centre for Epidemiology and Evidence, NSW Ministry of Health
The University of Sydney Page 2
What did we do?
• Updated Evidence Check review
• Included studies 2009-2015
– Strategies for human service agencies
– Implemented and evaluated
– Factors likely to influence use
• Overall
– 304 studies
– 187 primary studies
– 14 studies testing strategies
– Level of evidence weak
– Conclusions tentative
Moore, G., Redman, S., Haines, M. and Todd, A. 2011 What works to increase the use of research in population health policy and programs: A review,
Evidence & Policy, 7 (3): 251-279.
The University of Sydney Page 3
Study types
Study type n %
Studies testing strategies * 14 4.6
Qualitative methods (e.g. surveys, interviews) * 109 35.9
Literature reviews 38 12.5
Descriptive case studies * 64 21.1
Commentaries and opinion 69 22.7
Study designs and protocols 10 3.3
TOTAL 304 100.0
* focus of our analysis: 187 primary research
studies
Huw TO Davies, Alison E Powell, and Sandra M Nutley. 2015 Mobilising knowledge to improve UK health care: learning from other
countries and other sectors – a multimethod mapping study. Health Services and Delivery Research, No. 3.27
The University of Sydney Page 4
Reporting under five themes (all papers)
1. Relevant, useful, accessible research
2. Interaction and partnership research
3. Increasing organisational capacity
4. Research funding and infrastructure
5. Research priority setting
The University of Sydney Page 5
1 Relevant useful research (n=4)
New strategies tested:
 A writing tool
 Using policy briefs
 Seconding policymakers
 Commissioning rapid
reviews
Builds on knowledge:
 Access to websites
 Skills to access, appraise,
apply
New uses of known
factors:
• Flexible, fit for purpose
• Systems for rapid
access
• Knowledge brokers
• Interaction
The University of Sydney Page 6
1. In a nutshell….
Tailored, targeted research that is
fit for a specific purpose, audience
and context:
• Defining research needs
• Ensuring rapid access
• User friendly formats
• Integrating local / linked
data
The University of Sydney Page 7
2 Interaction, relationships, partnerships (n=5)
New strategies tested:
 Regular planned
interaction
 Networks
 Seconding policymakers
 Conference technology
Builds on knowledge:
 The use of forums and
workshops
Factors that influence use:
• Infrastructure
• Deliberative processes
• Expectations are a match
• Knowledge brokers /
facilitators
The University of Sydney Page 8
2. In a nutshell….
Confirmed the need for strategies to support partnerships and the
value of sustained engagement. Regular and ongoing interaction
continues to be a focus.
• Managers as intermediaries
• Clear expectations and ongoing communication
• Planned seminar series, networks and facilitated
exchanges
The University of Sydney Page 9
3 Increasing organisational capacity (n=4)
New strategies tested:
 Organisation-wide
capacity development
 Individual capacity
building
Builds on knowledge:
 Capacity building of
public health
professionals
Factors that influence use:
• Clear roles
• Infrastructure to support
interaction
• Managers and champions
The University of Sydney Page 10
3. In a nutshell….
Continued focus on individual and organisational level capacity
building, with some increases in knowledge and skills.
Organisational change harder to sustain.
• Clear expectations
• Infrastructure to support interaction / collaboration
• Policymakers, program managers and researchers as
brokers
The University of Sydney Page 11
4 Funding research and infrastructure (n=1)
New strategy tested:
 Grant-funded
collaboration with
policymakers,
practitioners and
university departments
Factors that influence use:
 Knowledge brokers and
intermediaries
The University of Sydney Page 12
4. In a nutshell….
Funded intermediary organisations to support linkage and
exchange, generate new research and change policy and practice.
• Required significant organisational support
• Deliberative processes lacking
The University of Sydney Page 13
Reflection: Where are we going?
• Knowledge brokers,
facilitators, champions
• Deliberative processes
• Large scale
organisational strategies
The University of Sydney Page 14
References
1. Brennan, S., et al. (2015). "Design and formative evaluation of the Policy Liaison Initiative: a long-term
knowledge translation strategy to encourage and support the use of Cochrane systematic reviews for informing
health policy." Evidence & Policy: A Journal of Research, Debate and Practice. [Theme 1]
2. Brownson, R. C., et al. (2011). "Communicating evidence-based information on cancer prevention to state-level
policy makers." J Natl Cancer Inst 103(4): 306-316. [Theme 1]
3. Bullock, A., et al. (2012). "Collaboration between health services managers and researchers: making a
difference?" Journal of health services research & policy 17(suppl 2): 2-10. [Theme 2]
4. Campbell, D., et al. (2011). "Evidence Check: knowledge brokering to commission research reviews for policy."
Evidence & Policy: A Journal of Research, Debate and Practice 7(1): 97-107. [Theme 1]
5. Dilworth, K., et al. (2013). "Making health promotion evidenced-informed: an organizational priority." Health
Promotion Practice 14(1): 139-145. [Theme 3]
6. Dwan, K. M., et al. (2015). "Measuring the success of facilitated engagement between knowledge producers and
users: a validated scale." Evidence & Policy: A Journal of Research, Debate and Practice 11(2): 239-252.
[Theme 2]
7. Hoeijmakers, M., et al. (2013). "Academic Collaborative Centre Limburg: a platform for knowledge transfer and
exchange in public health policy, research and practice?" Health Policy 111(2): 175-183. [Theme 4]
8. Househ, M. S., et al. (2011). "The use of conferencing technologies to support drug policy group knowledge
exchange processes: an action case approach." Int J Med Inform 80(4): 251-261. [Theme 3]
9. Jansen, M. W. and M. Hoeijmakers (2013). "A masterclass to teach public health professionals to conduct
practice-based research to promote evidence-based practice: a case study from The Netherlands." J Public
Health Manag Pract 19(1): 83-92. [Theme 3]
10. Kothari, A., et al. (2014). "Using an integrated knowledge translation approach to build a public health research
agenda." Health Research Policy & Systems 12: 6. [Theme 2]
11. Morris, Z. S., et al. (2013). "Developing engagement, linkage and exchange between health services managers
and researchers: Experience from the UK." Journal of health services research & policy 18(1 suppl): 23-29.
[Theme 2]
12. van der Heide, I., et al. (2015). "Implementation of a tool to enhance evidence-informed decision making in public
health: identifying barriers and facilitating factors." Evidence & Policy: A Journal of Research, Debate and
Practice. [Theme 1]
13. Wathen, C. N., et al. (2011). "Talk, trust and time: a longitudinal study evaluating knowledge translation and
exchange processes for research on violence against women." Implementation Science 6: 102. [Theme 2]
14. Yost, J., et al. (2014). "Evaluating the impact of an intensive education workshop on evidence-informed decision

How to boost policy and program agencies’ use of research

  • 1.
    The University ofSydney Page 1 Increasing the use of research in policy agencies: what works?Presented by Gabriel Moore School of Public Health University of Sydney and the Sax Institute Danielle Campbell Centre for Epidemiology and Evidence, NSW Ministry of Health
  • 2.
    The University ofSydney Page 2 What did we do? • Updated Evidence Check review • Included studies 2009-2015 – Strategies for human service agencies – Implemented and evaluated – Factors likely to influence use • Overall – 304 studies – 187 primary studies – 14 studies testing strategies – Level of evidence weak – Conclusions tentative Moore, G., Redman, S., Haines, M. and Todd, A. 2011 What works to increase the use of research in population health policy and programs: A review, Evidence & Policy, 7 (3): 251-279.
  • 3.
    The University ofSydney Page 3 Study types Study type n % Studies testing strategies * 14 4.6 Qualitative methods (e.g. surveys, interviews) * 109 35.9 Literature reviews 38 12.5 Descriptive case studies * 64 21.1 Commentaries and opinion 69 22.7 Study designs and protocols 10 3.3 TOTAL 304 100.0 * focus of our analysis: 187 primary research studies Huw TO Davies, Alison E Powell, and Sandra M Nutley. 2015 Mobilising knowledge to improve UK health care: learning from other countries and other sectors – a multimethod mapping study. Health Services and Delivery Research, No. 3.27
  • 4.
    The University ofSydney Page 4 Reporting under five themes (all papers) 1. Relevant, useful, accessible research 2. Interaction and partnership research 3. Increasing organisational capacity 4. Research funding and infrastructure 5. Research priority setting
  • 5.
    The University ofSydney Page 5 1 Relevant useful research (n=4) New strategies tested:  A writing tool  Using policy briefs  Seconding policymakers  Commissioning rapid reviews Builds on knowledge:  Access to websites  Skills to access, appraise, apply New uses of known factors: • Flexible, fit for purpose • Systems for rapid access • Knowledge brokers • Interaction
  • 6.
    The University ofSydney Page 6 1. In a nutshell…. Tailored, targeted research that is fit for a specific purpose, audience and context: • Defining research needs • Ensuring rapid access • User friendly formats • Integrating local / linked data
  • 7.
    The University ofSydney Page 7 2 Interaction, relationships, partnerships (n=5) New strategies tested:  Regular planned interaction  Networks  Seconding policymakers  Conference technology Builds on knowledge:  The use of forums and workshops Factors that influence use: • Infrastructure • Deliberative processes • Expectations are a match • Knowledge brokers / facilitators
  • 8.
    The University ofSydney Page 8 2. In a nutshell…. Confirmed the need for strategies to support partnerships and the value of sustained engagement. Regular and ongoing interaction continues to be a focus. • Managers as intermediaries • Clear expectations and ongoing communication • Planned seminar series, networks and facilitated exchanges
  • 9.
    The University ofSydney Page 9 3 Increasing organisational capacity (n=4) New strategies tested:  Organisation-wide capacity development  Individual capacity building Builds on knowledge:  Capacity building of public health professionals Factors that influence use: • Clear roles • Infrastructure to support interaction • Managers and champions
  • 10.
    The University ofSydney Page 10 3. In a nutshell…. Continued focus on individual and organisational level capacity building, with some increases in knowledge and skills. Organisational change harder to sustain. • Clear expectations • Infrastructure to support interaction / collaboration • Policymakers, program managers and researchers as brokers
  • 11.
    The University ofSydney Page 11 4 Funding research and infrastructure (n=1) New strategy tested:  Grant-funded collaboration with policymakers, practitioners and university departments Factors that influence use:  Knowledge brokers and intermediaries
  • 12.
    The University ofSydney Page 12 4. In a nutshell…. Funded intermediary organisations to support linkage and exchange, generate new research and change policy and practice. • Required significant organisational support • Deliberative processes lacking
  • 13.
    The University ofSydney Page 13 Reflection: Where are we going? • Knowledge brokers, facilitators, champions • Deliberative processes • Large scale organisational strategies
  • 14.
    The University ofSydney Page 14 References 1. Brennan, S., et al. (2015). "Design and formative evaluation of the Policy Liaison Initiative: a long-term knowledge translation strategy to encourage and support the use of Cochrane systematic reviews for informing health policy." Evidence & Policy: A Journal of Research, Debate and Practice. [Theme 1] 2. Brownson, R. C., et al. (2011). "Communicating evidence-based information on cancer prevention to state-level policy makers." J Natl Cancer Inst 103(4): 306-316. [Theme 1] 3. Bullock, A., et al. (2012). "Collaboration between health services managers and researchers: making a difference?" Journal of health services research & policy 17(suppl 2): 2-10. [Theme 2] 4. Campbell, D., et al. (2011). "Evidence Check: knowledge brokering to commission research reviews for policy." Evidence & Policy: A Journal of Research, Debate and Practice 7(1): 97-107. [Theme 1] 5. Dilworth, K., et al. (2013). "Making health promotion evidenced-informed: an organizational priority." Health Promotion Practice 14(1): 139-145. [Theme 3] 6. Dwan, K. M., et al. (2015). "Measuring the success of facilitated engagement between knowledge producers and users: a validated scale." Evidence & Policy: A Journal of Research, Debate and Practice 11(2): 239-252. [Theme 2] 7. Hoeijmakers, M., et al. (2013). "Academic Collaborative Centre Limburg: a platform for knowledge transfer and exchange in public health policy, research and practice?" Health Policy 111(2): 175-183. [Theme 4] 8. Househ, M. S., et al. (2011). "The use of conferencing technologies to support drug policy group knowledge exchange processes: an action case approach." Int J Med Inform 80(4): 251-261. [Theme 3] 9. Jansen, M. W. and M. Hoeijmakers (2013). "A masterclass to teach public health professionals to conduct practice-based research to promote evidence-based practice: a case study from The Netherlands." J Public Health Manag Pract 19(1): 83-92. [Theme 3] 10. Kothari, A., et al. (2014). "Using an integrated knowledge translation approach to build a public health research agenda." Health Research Policy & Systems 12: 6. [Theme 2] 11. Morris, Z. S., et al. (2013). "Developing engagement, linkage and exchange between health services managers and researchers: Experience from the UK." Journal of health services research & policy 18(1 suppl): 23-29. [Theme 2] 12. van der Heide, I., et al. (2015). "Implementation of a tool to enhance evidence-informed decision making in public health: identifying barriers and facilitating factors." Evidence & Policy: A Journal of Research, Debate and Practice. [Theme 1] 13. Wathen, C. N., et al. (2011). "Talk, trust and time: a longitudinal study evaluating knowledge translation and exchange processes for research on violence against women." Implementation Science 6: 102. [Theme 2] 14. Yost, J., et al. (2014). "Evaluating the impact of an intensive education workshop on evidence-informed decision

Editor's Notes

  • #4 Overall 304 studies 187 primary studies 14 studies testing strategies Level of evidence weak Conclusions tentative