AMREP Interactive Seminar 4th April 2012Bridging the “Know-Do” Gap: Communicating Results and Measuring Influence         ...
•   Increased focus on evidence-informed decision-making (EIDM) in the    public sector•   Increased pressure for research...
Originated from evidence-based medicine    “The integration of best research evidence    with clinical expertise and patie...
ResourcesDavies P. Workforce development to support evidence-informed public health: CuttingEdge Debates; 2005; VicHealth,...
Evidence for decision making – beyond clinical practiceSettings:Communities, public spaces, physical  and policy environme...
http://www.evidencesoup.com/canopener/Type 1: Data  To identify a problem / priority  e.g. What’s the prevalence of child ...
Interventions informed by research evidence are• More likely to be effective•   Reduce the likelihood for harm     – “Scar...
• Knowledge translation & exchange is about ensuring:   – stakeholders are aware of and use research evidence to     infor...
Knowledge translation is not linear...Image from: http://www.harlem-school.com/10TH/math_pdf/math.html
Example:US Centre for Disease Control & Prevention (Violence Prevention Branch)
Example:  Victorian government (Department of Health)Bowen S, Zwi AB (2005) Pathways to “Evidence-Informed”Policy and Prac...
Barriers to EIDM• International and local research with decision-makers and  practitioners suggest barriers are a lack of ...
Barriers for LMICs• Widest gap between information needed and what is available   – potential effectiveness of interventio...
KT&E strategies• Push   – Generally linked to dissemination strategies e.g. Systematic     reviews, clearinghouses, websit...
Strategies to support EIDM•   Clearinghouses•   Reviews/evidence summaries•   Systematic reviews•   Teaching programs•   K...
KT&E – evidence from clinical practice• Passive dissemination of information (e.g. didactic lectures,  recommendations for...
KT&E – evidence from Public health• Limited evidence in non-clinical contexts• One study in Canada.{Dobbins, 2009 }    – T...
KT&E: how evidence is used
KT&E: how evidence is used• Several types of research evidence utilisation can occur   – Instrumental       • Using inform...
KT4LG: Knowledge translation for local government • Aim: KT4LG is an intervention designed to increase the use of research...
Study preparation• Cochrane systematic review of effectiveness of knowledge  translation for public health decision-makers...
Background KT research•   The systematic review identified only one rigorous study evaluating the    effectiveness of a KT...
KT&E program logic – for EIDM in local government (KT4LG intervention)                                                    ...
Study componentsIntervention councils receive          2-year implementation    – Components 1 & 2Comparison councils rece...
Facilitated program componentsProgram of activities aim to support evidence-informed decision-making:
program toolsThe ‘evidence-informed process’3. Ask an answerable question4. Find the evidence to answer that question5. Cr...
program tools   Information derived from researchAdapted from: Swinburn B, Gill T and Kumanyika, S. Obesity prevention: A ...
program tools        Evidence ‘toolkit’Health Evidence Canada                      http://health-evidence.ca/        •   S...
program toolsApplicability & Transferability of research interventions                            http://www.nphp.gov.au/p...
Process evaluation•   To determine/describe reach, dose, adoption, intervention quality, fidelity…Tailored support (monthl...
Methodological challenges• What should primary and secondary outcome measures of KT be?   – Skills   – Confidence   – Orga...
Contextual challenges• Multi-sector approach• Diversity of expertise in research evidence use• Transient workforce – high ...
Health Evidence Canada              http://health-evidence.ca/   •    Searchable online registry of current               ...
Upcoming SlideShare
Loading in …5
×

Thinking through ‘knowledge translation’

867 views

Published on

Professor Elizabeth Waters, Coordinating Editor of the Cochrane Public Health Review Group & Melbourne School of Population Health, University of Melbourne

Published in: Health & Medicine, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
867
On SlideShare
0
From Embeds
0
Number of Embeds
127
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Notes from Liz: Orient to practitioners in an international development context and burnet international health academics/clin/ph folk. Many of them have come from UN agencies but interested in kt strategies on the ground. Be good to demonstrate understanding of spectrum of backgrounds of practitioners and issues around evidence for LMiC.
  • Bit of history of evidence-informed PH – essentially derived from evidence-based medicine– from medical field, but this is scientific and perhaps suggests that integrating evidence with professional expertise and situation is straightforward. Show definition of EIPH – we like the definition because it is more of a process – describes program development using reasoning, data and theory (BROWNSON) and again a process of integrating community needs with evidence-based interventions (KOHATSU).
  • This model builds onto the EBP model that we showed in the previous slide. Multiple factors are involved in the EIPH decision-making process – indeed these are all the different types of evidence that need to be incorporated into decisions about PH. Research evidence is what we will continue to discuss today – it is one part of the decision-making puzzle.
  • EIPH differs from EBP/EBM.... The obvious difference between EBM and EIPH is in: - The different settings (for interventions) where evidence can be applied – EBM tends to occur in clinical settings whereas PH relates to whole communities and public policy environments. Also the uses of evidence vary – EBP tends to relate to decisions about individuals (e.g. Treatment), whereas in PH is for decisions applicable to entire communities, such as for town planning, policy development, service delivery, facility development. The influences are also different – refer to the previous slide - Last the types of evidence will vary – EBP is often limited and refer to heirarchy of evidence whereas PH is broader – we will discuss this in a moment….
  • In our KT research group, we broadly conceptualise “evidence”. This is not definitive but offers a useful summary. Using ideas from Brownson and colleagues – three types 1 – data – prevalence of issue , a priority – example: 2 – effectiveness – what do we DO about the problem – example: 3 – implementation – qual , quant- helps with transfer of findings into your own setting, understand community barriers/enablers to an intervention  
  • I am possibly preaching to the converted when we ask why being ‘evidence-informed’ is important. Although the outcome is perhaps not life and death like making a decision in a clinical context, there are perhaps implications of not practicing in an evidence-informed way in health promotion & public health. Reduce likelihood of harm – this is important considering programs/policies that continued, sometimes for decades, until rigorous and appropriate evaluations revealed that the results were not as intended. Resource allocation – more effective, equitable Focus efforts – target ints, improve reach. Acccountability: have documentation of program impact, for example -
  • But how do we ‘be evidence-informed ’? This brings us to the concept of KT
  • We know that transferring research interventions  practice is not straightforward or linear, it is cyclical – no magic equation for achieving EIDM
  • International example – KT in the CDC. CDC Expert review panel to discuss how to incorporate the knowledge, expertise and evidence from the practitioners and policy makers – as you can see, this is similar to the EBP model before.
  • Local example – KT in Department of Health Victoria. A descriptive theoretical model of the pathway (steps) involved in using evidence in policy.
  • This list comes from the published literature on barriers to research use in organizations, plus experiential info from practitioners in our partnership work
  • Evidence gaps in LMICs
  • So, what do we know about whether KT strategies have any impact on EIDM?
  • As you are all well aware there are a number of ways that evidence can be used (or generated) in practice. Broadly we can categorize into direct and indirect. Directly to develop and plan, create strategy. Indirectly to create a culture. This presents challenges for measuring impact and outcome.
  • CULL? – could talk to points in previous slide
  • CULL
  • CULL? – i like this slide but repetitive of previous – could keep this one and delete previous
  • CULL
  • CULL
  • CULL
  • CULL
  • CULL – or is it worth having one slide on evaluation?
  • Here are some examples of websites aimed at facilitating EIDM (push) But there are barriers to applying SR findings to policy and practice
  • Thinking through ‘knowledge translation’

    1. 1. AMREP Interactive Seminar 4th April 2012Bridging the “Know-Do” Gap: Communicating Results and Measuring Influence Professor Elizabeth Waters Jack Brockhoff Chair of Child Public Health Director, Jack Brockhoff Child Health and Wellbeing Program ; McCaughey Centre, Melbourne School of Population Health, University of Melbourne
    2. 2. • Increased focus on evidence-informed decision-making (EIDM) in the public sector• Increased pressure for researchers to ‘translate’ or ‘transfer’ their findings• Increased focus on the need for a more effective policy-practice-research interface – Some new infrastructure Internationally and locally aiming to support this• Potentially effective strategies – Evidence summaries – Knowledge brokering – Training – Networking/info-sharing
    3. 3. Originated from evidence-based medicine “The integration of best research evidence with clinical expertise and patient values”Sackett et al, 1996 BMJ 312 : 71
    4. 4. ResourcesDavies P. Workforce development to support evidence-informed public health: CuttingEdge Debates; 2005; VicHealth, Melbourne.
    5. 5. Evidence for decision making – beyond clinical practiceSettings:Communities, public spaces, physical and policy environments… Uses: Program, service and policy Interventions: development, land-use planning… Policies and regulations Organisational development Community development Engineering , technical developments Service development and delivery Communication, social marketing Programs
    6. 6. http://www.evidencesoup.com/canopener/Type 1: Data To identify a problem / priority e.g. What’s the prevalence of child dental caries in the community?Type 2: Effectiveness Helps to identify ‘what works’ to address a problem e.g. Can poor health be improved by improving housing?Type 3: Implementation Helps to understand ‘what works, for whom, in what context, and why’ e.g. Will a mainstream early learning program be applicable to the aboriginal community? Adapted from Brownson et al. 2009
    7. 7. Interventions informed by research evidence are• More likely to be effective• Reduce the likelihood for harm – “Scared straight” crime prevention program – increased offences – Early versions of the DARE substance abuse prevention program – “Bike Ed” to reduce cycle injuries – significantly increased injuries among boys – Toughened pint glasses in pubs – significantly increased injury rate – Sesame Street – increased learning overall but also increased inequality gap between slow and fast learners R Vaughan (2004) American Journal of Public Health• Better use of limited resources (Time, money)• Better cost-effectiveness/cost-benefit of investment – Playground safety floor-coverings – 20% increase in injuries, high cost (40% of total) T Gill (2011) Rethinking childhood – presentation to VicHealth, 2011• Help to focus efforts, improve reach• Provide ‘back up’ for implementation – accountability
    8. 8. • Knowledge translation & exchange is about ensuring: – stakeholders are aware of and use research evidence to inform their decision making – research is informed by current available evidence and the experience and information needs of stakeholders• Growing awareness among – practitioners/policy-makers about systematic reviews; – researchers of the need for policy-relevant research; – Challenge: how to support use of research and reviews in informing decisions
    9. 9. Knowledge translation is not linear...Image from: http://www.harlem-school.com/10TH/math_pdf/math.html
    10. 10. Example:US Centre for Disease Control & Prevention (Violence Prevention Branch)
    11. 11. Example: Victorian government (Department of Health)Bowen S, Zwi AB (2005) Pathways to “Evidence-Informed”Policy and Practice: A Framework for Action. PLoS Med 2(7)
    12. 12. Barriers to EIDM• International and local research with decision-makers and practitioners suggest barriers are a lack of • Time • Trust • Access to evidence • Accessibility of evidence • Access to researchers • Timeliness • Evidence
    13. 13. Barriers for LMICs• Widest gap between information needed and what is available – potential effectiveness of interventions to improve health in LMICs remains untested (Buekens 2004)• Few global interventions targeting priority issues specific to LMICs are evidence based (other than vaccination) – Routine interventions not fully supported by good data e.g. handwashing, large scale nutrition education programs• Interventions shown to be effective HICs have not been shown to be similarly effective in LMIC context (Buekens 2004; Ebrahim 2005)
    14. 14. KT&E strategies• Push – Generally linked to dissemination strategies e.g. Systematic reviews, clearinghouses, websites• Pull – Support DMs to seek out evidence e.g. Workforce development, IT access, DM tools• Exchange – Meaningful and reciprocal relationship
    15. 15. Strategies to support EIDM• Clearinghouses• Reviews/evidence summaries• Systematic reviews• Teaching programs• Knowledge brokering• Partnerships/stakeholder engagement
    16. 16. KT&E – evidence from clinical practice• Passive dissemination of information (e.g. didactic lectures, recommendations for clinical care) – Found to be generally ineffective{Bero, 1998 }• Directed strategies (e.g. educational outreach, reminders, educational meetings) – Found to be effective – More recent reviews have supported these findings{Grimshaw JM, 2001 ; Grimshaw, 2002 ; Grimshaw, 2004 }• Educational meetings either delivered alone or in combination with, local opinion leaders, and multidisciplinary committees – Emerging evidence {Thompson, 2007 }
    17. 17. KT&E – evidence from Public health• Limited evidence in non-clinical contexts• One study in Canada.{Dobbins, 2009 } – Tailored messaging optimal to knowledge brokering or access to an online registry of systematic reviews – Tailored messages most effective in organizations who were perceived as having a high research culture – Knowledge brokers potentially effective in organizations who were perceived as having low research culture• Supplementary evidence from uncontrolled studies – interactive strategies – promising interventions worthy of further exploration – Studies exploring the utility of evidence syntheses also identified the benefits of interaction in developing topics and their subsequent use in policy and practice
    18. 18. KT&E: how evidence is used
    19. 19. KT&E: how evidence is used• Several types of research evidence utilisation can occur – Instrumental • Using information directly to inform an action, policy, proposal etc • Can be measured e.g. Tracer studies, alignment – Conceptual • Using the information indirectly for general enlightenment • Hard to capture – Symbolic • Using information to support an existing action, policy, program • Hard to capture – doesn’t consider the broader evidence on a topic• How can we reasonably measure them? – Outcomes – individual vs organisational research use
    20. 20. KT4LG: Knowledge translation for local government • Aim: KT4LG is an intervention designed to increase the use of research evidence to support evidence-informed decision making in local govts • Background research conducted to plan the intervention • Logic model developed informed by background research • Intervention comprises • Tailored support by a program coordinator • Group training • Targeted communications • Objective is to assist local governments access, assess adapt & apply research evidence to local decisions related to public health planning and implementation • was focused on obesity prevention – evolved over time
    21. 21. Study preparation• Cochrane systematic review of effectiveness of knowledge translation for public health decision-makers• Review of literature and theory• Baseline survey of all Victorian Local Governments (79) – 45 councils participated• Key informant interviews with sample of local government planners• Consideration of local government role/ capacity to act
    22. 22. Background KT research• The systematic review identified only one rigorous study evaluating the effectiveness of a KT intervention in a public health setting• Theory and literature review showed promising KT strategies/conceptual frameworks• The state-wide EIDM survey and interviews highlighted the importance of – access – a skilled workforce and – the influence of organisational culture on EIDM• Suggested strategies to support EIDM for public health in local government included – skills development – provision of resources and tools and – networking for information sharing within and between councils
    23. 23. KT&E program logic – for EIDM in local government (KT4LG intervention) OUTCOMES OUTPUTS ACTIVITIES / PARTICIPATION/REACH Individuals Organisation System COMPONENTS Increase Improve APPROACH: Knowledge Utilisation Framework / confidence EIDM culture within EIDM culture and Diffusion of Innovations PH team system-level support Tailored organisational Leads to support Intervention Increase Increased research use within councils: skills PH teamFacilitated Nominated (instrumentalprogram Group Training Session LG staff within and conceptual) PH related departments And this Increase could lead to Targeted communication EXPLORE: What is the access role of the organisation and system and Improve how does it need EIDM culture within to be changed? organisation team Comparison Increase And then Evidence Summaries councils access this… Increased research use across Assumptions organisation Increased support and interaction through the facilitated program will improve (instrumental outcomes and conceptual) Knowledge, confidence and skills predict research use EIDM culture at the system level influences the EIDM culture and research use in organisations but requires individuals with knowledge, skills and confidence Innovation will spread through organisations and the system Contextual Factors Type of decision, type of use Process Evaluation Decision making structures / systems Programme reach / Dose / Fidelity / Quality/ Cost Council size and structure
    24. 24. Study componentsIntervention councils receive 2-year implementation – Components 1 & 2Comparison councils receive – Component 1 onlyComponent 1: Research evidence summaries• Prepared and disseminated a series of summaries outlining possible intervention options for obesity prevention at local government level• Review of systematic reviewsComponent 2: Facilitated program• Regular contact with program coordinator• Program of activities that aim to support evidence-informed decision- making including training and networking events
    25. 25. Facilitated program componentsProgram of activities aim to support evidence-informed decision-making:
    26. 26. program toolsThe ‘evidence-informed process’3. Ask an answerable question4. Find the evidence to answer that question5. Critically appraise the evidence6. Integrate the evidence with your expertise and values of population7. Evaluate your effectiveness in evidence-informed decision makingAdapted from:Dawes et al (2005) Sicily Statement on Evidence-Based Practice, BMC Medical education 2005, 5:1
    27. 27. program tools Information derived from researchAdapted from: Swinburn B, Gill T and Kumanyika, S. Obesity prevention: A proposed framework for translating evidence into action. Obesity Reviews, 2005; 6(1), pp. 23-33.
    28. 28. program tools Evidence ‘toolkit’Health Evidence Canada http://health-evidence.ca/ • Searchable online registry of current research evidence (Canada)Cochrane Collaboration www.thecochranelibrary.com • Searchable online registry of systematic www.ph.cochrane.org reviews (Global)The National Library for Public Health http://www2.evidence.nhs.uk/se Searchable collection of research evidence(NLPH), National Institute for Health and arch-and-browse and public health guidance (UK)Clinical Evidence (NICE)Centre for Reviews and Dissemination www.york.ac.uk/inst/crd/ • Online database of systematic reviews, health economics research, abstracts and health technology assessments (UK)The Campbell Collaboration www.campbellcollaboration.org/ • Online database of systematic reviews: education, crime and justice, social welfare (Global)Sax Institute www.saxinstitute.org.au • Online access to rapid reviews (‘evidence checks’) (Australia)Guide to Community Preventive Services www.thecommunityguide.org Searchable online registry of systematic(The Guide) reviews (USA)The Evidence for Policy and Practice www.eppi.ioe.ac.uk Online ‘evidence library’ of systematicInformation (EPPI) and Co-ordinating reviews (UK)CentreEffective Public Health Practice Project www.ephpp.ca/ourwork.html Online access to systematic reviews,(EPHPP) summary statements and tools (Canada)Department of Health, Victoria www.health.vic.gov.au/healthpro Evidence-based resources (Victoria, motion/evidence_res/evidence_in Australia) dex
    29. 29. program toolsApplicability & Transferability of research interventions http://www.nphp.gov.au/publications/rd/schemaV4.pdf
    30. 30. Process evaluation• To determine/describe reach, dose, adoption, intervention quality, fidelity…Tailored support (monthly contacts)• Interactions with each participant captured, database maintained by program coordinator – Status of participation, role in council – Type of contact e.g. to discuss research use; support request; relationship-building – Topics e.g. Food security, physical activity, community gardens, alcohol licensing… – Focus e.g. Types of evidence, instrumental or conceptual use, skills… – Process e.g. action planning, strategy development, evaluation… – KT focus e.g. promoting instrumental use, confidence, access, organisational culture…Group training• Attendance (individuals, councils); attendee experiences (qualitative)• Reflective practice journal
    31. 31. Methodological challenges• What should primary and secondary outcome measures of KT be? – Skills – Confidence – Organisational culture – Use of research evidence• How do we measure these? – What are the questions we need to ask? – What are the best methods for evaluating these outcomes • Audits, self report survey, key informant interviews, diary – Can we develop a validated tool to measure evidence-uptake?
    32. 32. Contextual challenges• Multi-sector approach• Diversity of expertise in research evidence use• Transient workforce – high turnover and changing roles• ‘Noisy environment’ – many public health interventions, new funding, change in priorities and scope of LG work• Level of priority to topic differs across LGs• Limited evidence in some topic areas, ‘too much’ in others• Politics of decision-making – research evidence is but one part of the puzzle
    33. 33. Health Evidence Canada http://health-evidence.ca/ • Searchable online registry of current research evidence (Canada)Cochrane Collaboration www.thecochranelibrary.com • Searchable online registry of www.ph.cochrane.org systematic reviews (Global)The National Library for Public http://www2.evidence.nhs.uk Searchable collection of researchHealth (NLPH), National Institute for /search-and-browse evidence and public health guidanceHealth and Clinical Evidence (NICE) (UK)Centre for Reviews and www.york.ac.uk/inst/crd/ • Online database of systematicDissemination reviews, health economics research, abstracts and health technology assessments (UK)The Campbell Collaboration www.campbellcollaboration. • Online database of systematic org/ reviews: education, crime and justice, social welfare (Global)Sax Institute www.saxinstitute.org.au • Online access to rapid reviews (‘evidence checks’) (Australia)Guide to Community Preventive www.thecommunityguide.org Searchable online registry ofServices (The Guide) systematic reviews (USA)The Evidence for Policy and Practice www.eppi.ioe.ac.uk Online ‘evidence library’ of systematicInformation (EPPI) and Co- reviews (UK)ordinating CentreEffective Public Health Practice www.ephpp.ca/ourwork.html Online access to systematic reviews,Project (EPHPP) summary statements and tools (Canada)

    ×