Professor Elizabeth Waters, Coordinating Editor of the Cochrane Public Health Review Group & Melbourne School of Population Health, University of Melbourne

1. AMREP Interactive Seminar 4th April 2012
Bridging the “Know-Do” Gap: Communicating Results and Measuring Influence
Professor Elizabeth Waters
Jack Brockhoff Chair of Child Public Health
Director, Jack Brockhoff Child Health and
Wellbeing Program ; McCaughey Centre,
Melbourne School of Population Health,
University of Melbourne

2. • Increased focus on evidence-informed decision-making (EIDM) in the
public sector
• Increased pressure for researchers to ‘translate’ or ‘transfer’ their findings
• Increased focus on the need for a more effective policy-practice-research
interface
– Some new infrastructure Internationally and locally aiming to support this
• Potentially effective strategies
– Evidence summaries
– Knowledge brokering
– Training
– Networking/info-sharing

3. Originated from evidence-based medicine
“The integration of best research evidence
with clinical expertise and patient values”
Sackett et al, 1996 BMJ 312 : 71

4. Resources
Davies P. Workforce development to support evidence-informed public health: Cutting
Edge Debates; 2005; VicHealth, Melbourne.

5. Evidence for decision making – beyond clinical practice
Settings:
Communities, public spaces, physical
and policy environments… Uses:
Program, service and policy
Interventions: development,
land-use planning…
Policies and regulations
Organisational development
Community development
Engineering , technical
developments
Service development and
delivery
Communication, social
marketing
Programs

6. http://www.evidencesoup.com/canopener/
Type 1: Data
To identify a problem / priority
e.g. What’s the prevalence of child dental caries in the community?
Type 2: Effectiveness
Helps to identify ‘what works’ to address a problem
e.g. Can poor health be improved by improving housing?
Type 3: Implementation
Helps to understand ‘what works, for whom, in what context, and why’
e.g. Will a mainstream early learning program be applicable to the aboriginal
community?
Adapted from Brownson et al. 2009

7. Interventions informed by research evidence are
• More likely to be effective
• Reduce the likelihood for harm
– “Scared straight” crime prevention program – increased offences
– Early versions of the DARE substance abuse prevention program
– “Bike Ed” to reduce cycle injuries – significantly increased injuries among boys
– Toughened pint glasses in pubs – significantly increased injury rate
– Sesame Street – increased learning overall but also increased inequality gap between slow
and fast learners R Vaughan (2004) American Journal of Public Health
• Better use of limited resources (Time, money)
• Better cost-effectiveness/cost-benefit of investment
– Playground safety floor-coverings – 20% increase in injuries, high cost (40% of total)
T Gill (2011) Rethinking childhood – presentation to VicHealth, 2011
• Help to focus efforts, improve reach
• Provide ‘back up’ for implementation – accountability

8. • Knowledge translation & exchange is about ensuring:
– stakeholders are aware of and use research evidence to
inform their decision making
– research is informed by current available evidence and the
experience and information needs of stakeholders
• Growing awareness among
– practitioners/policy-makers about systematic reviews;
– researchers of the need for policy-relevant research;
– Challenge: how to support use of research and reviews in
informing decisions

9. Knowledge translation is not linear...
Image from: http://www.harlem-
school.com/10TH/math_pdf/math.html

10. Example:
US Centre for Disease Control & Prevention (Violence Prevention Branch)

11. Example:
Victorian government (Department of Health)
Bowen S, Zwi AB (2005) Pathways to “Evidence-Informed”Policy and Practice: A Framework for Action. PLoS Med 2(7)

12. Barriers to EIDM
• International and local research with decision-makers and
practitioners suggest barriers are a lack of
• Time
• Trust
• Access to evidence
• Accessibility of evidence
• Access to researchers
• Timeliness
• Evidence

13. Barriers for LMICs
• Widest gap between information needed and what is available
– potential effectiveness of interventions to improve health in LMICs
remains untested (Buekens 2004)
• Few global interventions targeting priority issues specific to
LMICs are evidence based (other than vaccination)
– Routine interventions not fully supported by good data e.g. handwashing,
large scale nutrition education programs
• Interventions shown to be effective HICs have not been shown
to be similarly effective in LMIC context (Buekens 2004; Ebrahim
2005)

14. KT&E strategies
• Push
– Generally linked to dissemination strategies e.g. Systematic
reviews, clearinghouses, websites
• Pull
– Support DMs to seek out evidence e.g. Workforce
development, IT access, DM tools
• Exchange
– Meaningful and reciprocal relationship

15. Strategies to support EIDM
• Clearinghouses
• Reviews/evidence summaries
• Systematic reviews
• Teaching programs
• Knowledge brokering
• Partnerships/stakeholder engagement

16. KT&E – evidence from clinical practice
• Passive dissemination of information (e.g. didactic lectures,
recommendations for clinical care)
– Found to be generally ineffective{Bero, 1998 }
• Directed strategies (e.g. educational outreach, reminders,
educational meetings)
– Found to be effective – More recent reviews have supported these
findings{Grimshaw JM, 2001 ; Grimshaw, 2002 ; Grimshaw, 2004 }
• Educational meetings either delivered alone or in combination
with, local opinion leaders, and multidisciplinary committees
– Emerging evidence {Thompson, 2007 }

17. KT&E – evidence from Public health
• Limited evidence in non-clinical contexts
• One study in Canada.{Dobbins, 2009 }
– Tailored messaging optimal to knowledge brokering or access to an online
registry of systematic reviews
– Tailored messages most effective in organizations who were perceived as having
a high research culture
– Knowledge brokers potentially effective in organizations who were perceived as
having low research culture
• Supplementary evidence from uncontrolled studies
– interactive strategies – promising interventions worthy of further exploration
– Studies exploring the utility of evidence syntheses also identified the benefits of
interaction in developing topics and their subsequent use in policy and practice

18. KT&E: how evidence is used

19. KT&E: how evidence is used
• Several types of research evidence utilisation can occur
– Instrumental
• Using information directly to inform an action, policy, proposal etc
• Can be measured e.g. Tracer studies, alignment
– Conceptual
• Using the information indirectly for general enlightenment
• Hard to capture
– Symbolic
• Using information to support an existing action, policy, program
• Hard to capture – doesn’t consider the broader evidence on a topic
• How can we reasonably measure them?
– Outcomes – individual vs organisational research use

20. KT4LG: Knowledge translation for local government
• Aim: KT4LG is an intervention designed to increase the use of research
evidence to support evidence-informed decision making in local govts
• Background research conducted to plan the intervention
• Logic model developed informed by background research
• Intervention comprises
• Tailored support by a program coordinator
• Group training
• Targeted communications
• Objective is to assist local governments access, assess adapt & apply
research evidence to local decisions related to public health planning
and implementation
• was focused on obesity prevention – evolved over time

21. Study preparation
• Cochrane systematic review of effectiveness of knowledge
translation for public health decision-makers
• Review of literature and theory
• Baseline survey of all Victorian Local Governments (79)
– 45 councils participated
• Key informant interviews with sample of local government
planners
• Consideration of local government role/
capacity to act

22. Background KT research
• The systematic review identified only one rigorous study evaluating the
effectiveness of a KT intervention in a public health setting
• Theory and literature review showed promising KT strategies/conceptual
frameworks
• The state-wide EIDM survey and interviews highlighted the importance of
– access
– a skilled workforce and
– the influence of organisational culture on EIDM
• Suggested strategies to support EIDM for public health in local government
included
– skills development
– provision of resources and tools and
– networking for information sharing within and between councils

23. KT&E program logic – for EIDM in local government (KT4LG intervention)
OUTCOMES
OUTPUTS
ACTIVITIES / PARTICIPATION/REACH Individuals Organisation System
COMPONENTS
Increase Improve
APPROACH: Knowledge Utilisation Framework / confidence EIDM culture within EIDM culture and
Diffusion of Innovations PH team system-level support
Tailored organisational Leads to
support Intervention Increase Increased research use within
councils: skills PH team
Facilitated Nominated (instrumental
program Group Training Session LG staff within and conceptual)
PH related
departments And this
Increase could lead to
Targeted communication EXPLORE: What is the
access role of the organisation
and system and
Improve how does it need
EIDM culture within to be changed?
organisation team
Comparison Increase
And then
Evidence Summaries councils access this…
Increased research use across
Assumptions organisation
Increased support and interaction through the facilitated program will improve (instrumental
outcomes and conceptual)
Knowledge, confidence and skills predict research use
EIDM culture at the system level influences the EIDM culture and research use in
organisations but requires individuals with knowledge, skills and confidence
Innovation will spread through organisations and the system
Contextual Factors
Type of decision, type of use
Process Evaluation Decision making structures / systems
Programme reach / Dose / Fidelity / Quality/ Cost Council size and structure

24. Study components
Intervention councils receive 2-year implementation
– Components 1 & 2
Comparison councils receive
– Component 1 only
Component 1: Research evidence summaries
• Prepared and disseminated a series of summaries outlining possible
intervention options for obesity prevention at local government level
• Review of systematic reviews
Component 2: Facilitated program
• Regular contact with program coordinator
• Program of activities that aim to support evidence-informed decision-
making including training and networking events

25. Facilitated program components
Program of activities aim to support evidence-informed decision-making:

26. program tools
The ‘evidence-informed process’
3. Ask an answerable question
4. Find the evidence to answer that question
5. Critically appraise the evidence
6. Integrate the evidence with your expertise and values of
population
7. Evaluate your effectiveness in evidence-informed decision
making
Adapted from:
Dawes et al (2005) Sicily Statement on Evidence-Based Practice, BMC Medical education 2005, 5:1

27. program tools
Information derived from research
Adapted from: Swinburn B, Gill T and Kumanyika, S. Obesity prevention: A proposed framework for translating evidence into
action. Obesity Reviews, 2005; 6(1), pp. 23-33.

28. program tools
Evidence ‘toolkit’
Health Evidence Canada http://health-evidence.ca/ • Searchable online registry of current
research evidence (Canada)
Cochrane Collaboration www.thecochranelibrary.com • Searchable online registry of systematic
www.ph.cochrane.org reviews (Global)
The National Library for Public Health http://www2.evidence.nhs.uk/se Searchable collection of research evidence
(NLPH), National Institute for Health and arch-and-browse and public health guidance (UK)
Clinical Evidence (NICE)
Centre for Reviews and Dissemination www.york.ac.uk/inst/crd/ • Online database of systematic reviews,
health economics research, abstracts and
health technology assessments (UK)
The Campbell Collaboration www.campbellcollaboration.org/ • Online database of systematic reviews:
education, crime and justice, social welfare
(Global)
Sax Institute www.saxinstitute.org.au • Online access to rapid reviews (‘evidence
checks’) (Australia)
Guide to Community Preventive Services www.thecommunityguide.org Searchable online registry of systematic
(The Guide) reviews (USA)
The Evidence for Policy and Practice www.eppi.ioe.ac.uk Online ‘evidence library’ of systematic
Information (EPPI) and Co-ordinating reviews (UK)
Centre
Effective Public Health Practice Project www.ephpp.ca/ourwork.html Online access to systematic reviews,
(EPHPP) summary statements and tools (Canada)
Department of Health, Victoria www.health.vic.gov.au/healthpro Evidence-based resources (Victoria,
motion/evidence_res/evidence_in Australia)
dex

29. program tools
Applicability & Transferability of research interventions
http://www.nphp.gov.au/publications/rd/schemaV4.pdf

30. Process evaluation
• To determine/describe reach, dose, adoption, intervention quality, fidelity…
Tailored support (monthly contacts)
• Interactions with each participant captured, database maintained by
program coordinator
– Status of participation, role in council
– Type of contact e.g. to discuss research use; support request; relationship-building
– Topics e.g. Food security, physical activity, community gardens, alcohol licensing…
– Focus e.g. Types of evidence, instrumental or conceptual use, skills…
– Process e.g. action planning, strategy development, evaluation…
– KT focus e.g. promoting instrumental use, confidence, access, organisational culture…
Group training
• Attendance (individuals, councils); attendee experiences (qualitative)
• Reflective practice journal

31. Methodological challenges
• What should primary and secondary outcome measures of KT be?
– Skills
– Confidence
– Organisational culture
– Use of research evidence
• How do we measure these?
– What are the questions we need to ask?
– What are the best methods for evaluating these outcomes
• Audits, self report survey, key informant interviews, diary
– Can we develop a validated tool to measure evidence-uptake?

32. Contextual challenges
• Multi-sector approach
• Diversity of expertise in research evidence use
• Transient workforce – high turnover and changing roles
• ‘Noisy environment’ – many public health interventions, new funding,
change in priorities and scope of LG work
• Level of priority to topic differs across LGs
• Limited evidence in some topic areas, ‘too much’ in others
• Politics of decision-making – research evidence is but one part of the
puzzle

33. Health Evidence Canada http://health-evidence.ca/ • Searchable online registry of current
research evidence (Canada)
Cochrane Collaboration www.thecochranelibrary.com • Searchable online registry of
www.ph.cochrane.org systematic reviews (Global)
The National Library for Public http://www2.evidence.nhs.uk Searchable collection of research
Health (NLPH), National Institute for /search-and-browse evidence and public health guidance
Health and Clinical Evidence (NICE) (UK)
Centre for Reviews and www.york.ac.uk/inst/crd/ • Online database of systematic
Dissemination reviews, health economics research,
abstracts and health technology
assessments (UK)
The Campbell Collaboration www.campbellcollaboration. • Online database of systematic
org/ reviews: education, crime and justice,
social welfare (Global)
Sax Institute www.saxinstitute.org.au • Online access to rapid reviews
(‘evidence checks’) (Australia)
Guide to Community Preventive www.thecommunityguide.org Searchable online registry of
Services (The Guide) systematic reviews (USA)
The Evidence for Policy and Practice www.eppi.ioe.ac.uk Online ‘evidence library’ of systematic
Information (EPPI) and Co- reviews (UK)
ordinating Centre
Effective Public Health Practice www.ephpp.ca/ourwork.html Online access to systematic reviews,
Project (EPHPP) summary statements and tools
(Canada)