Your SlideShare is downloading. ×
John Lavis | Making research work for decision makers: international perspectives
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

John Lavis | Making research work for decision makers: international perspectives

306
views

Published on

Professor John N Lavis, Director of the McMaster Health Forum at McMaster University in Canada, recently addressed a CIPHER forum to share his experience in making research useful for health decision …

Professor John N Lavis, Director of the McMaster Health Forum at McMaster University in Canada, recently addressed a CIPHER forum to share his experience in making research useful for health decision makers.

CIPHER, the Centre for Informing Policy in Health with Evidence from Research, is an Australian collaborative research centre managed by the Sax Institute, that is investigating the tools, skills and systems that might contribute to an increased use of research evidence in policy.

For more information visit www.saxinstitute.org.au.

Published in: Government & Nonprofit

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
306
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 1 International Perspectives Making Research Work for Decision Makers: Innovations and Priorities for Action Sydney, Australia John N. Lavis, MD, PhD Professor and Director, McMaster Health Forum McMaster University Adjunct Professor of Global Health Harvard School of Public Health
  • 2.  I am or was involved in  Creation and operation of the McMaster Health Forum (MHF)  Development and maintenance of Health Systems Evidence (HSE)  Development of (soon-to-launch) Health Systems Learning (HSL)  Work of WHO-sponsored Evidence-Informed Policy Networks (EVIPNet)  Development of the SUPPORT tools Conflict of Interest Statement 2
  • 3.  Rationales for and a definition of evidence-informed decision-making  Promising practices in addressing the challenges to finding and using research evidence  What is needed to do this work, with a detour about  Clarifying a problem  Framing options  Identifying implementation considerations  My top three priorities for action Overview 3
  • 4.  Which would be considered the most compelling in NSW / Australia?  Better process / more efficient process?  Better decisions / more acceptance of decisions?  Better communication of decisions / less embarrassment?  Better outcomes / better value for money?  Note that research evidence can be used in many ways  Solve particular problems at hand (instrumental)  Think about problems/options differently (conceptual)  Justify a decision made for other reasons (political) Possible Rationales for Evidence-Informed Decision-making 4
  • 5.  Using the best available* data and research evidence – systematically and transparently – in the time available in each of  Agenda setting (esp. clarifying the problem iteratively, while being attentive to policy and politics)  Policy or program development (esp. framing options iteratively)  Implementation (esp. identifying barriers / facilitators iteratively and strategies to address them) * Best available research evidence = highest quality, most locally applicable, synthesized research evidence (looking first for a perfect match to support an instrumental use and then looking more broadly to support a conceptual use) A Possible Definition of Evidence-Informed Decision-making 5
  • 6. 1. Research evidence competes with many other factors in the decision-making process 2. Research evidence isn’t valued as an information input 3. Research evidence isn’t relevant 4. Research evidence isn’t easy to use Caveat  The approaches I’ll describe are promising (based on a synthesis of findings from 124 observational studies) but not yet tested rigorously in terms of effectiveness Addressing the Challenges in Using Research Evidence 6
  • 7. Challenge 1  Research evidence competes with many other factors in the decision-making process • Institutional constraints • Interest group pressure • Other ideas such as citizens’ values, tacit knowledge, real-world views and experiences • External events (e.g., global recession) One option (among many) for addressing challenge 1  Improve decision-making processes (but this is beyond the scope of many of us) or create “routine” processes (as many countries have done for new technologies) (e.g., NICE) Addressing Challenge 1 7
  • 8. Challenge 2  Research evidence isn’t valued as an information input Two options (among many) for addressing challenge 2  Convince decision-makers and stakeholders to place value on the use of research evidence by highlighting examples from the past or from other jurisdictions where research evidence made the difference between policy/program success and failure (or between communication success and failure) (e.g., WHO)  Work with journalists to highlight statements that are and aren’t based on the best available research evidence (e.g., Science-ish) Addressing Challenge 2 8
  • 9. Challenge 3  Research evidence isn’t relevant One option (among many) for addressing challenge 3  Engage decision-makers and stakeholders periodically in priority- setting processes and communicate the priorities to researchers • Long-term requirements for data-collection systems and for new primary research (e.g., Listening for Direction) • Medium-term term requirements for systematic reviews (e.g., 3ie) • Short-term requirements for ‘evidence briefs’ (e.g., EVIPNet Cameroon) Addressing Challenge 3 9
  • 10. Challenge 4  Research evidence isn’t easy to use Challenge 4a  Research evidence isn’t communicated effectively (i.e., decision-makers and stakeholders hear noise instead of music) Two options (among many) for addressing challenge 4a  Identify a high-priority issue, identify data and systematic reviews that address different facets of the issue (problem, options and implementation considerations), and prepare an ‘evidence brief’ to summarize the findings (e.g., EVIPNet, MHF)  Stop (incentivizing) researchers and university media offices from publicizing the results of single studies without putting them in the context of a systematic review (and clarifying the many other questions that need to be answered before action is taken) Addressing Challenge 4a 10
  • 11. 11 Addressing Challenge 4a (2)
  • 12. Challenge 4  Research evidence isn’t easy to use Challenge 4b  Research evidence isn’t available when decision-makers and stakeholders need it and in a form that they can use Three options (among many) for addressing challenge 4b  Promote the use of ‘one-stop shops’ for synthesized research evidence • e.g., Cochrane Library and PubMed ‘hedges’ for ‘my health’ • e.g., Health Evidence and PubMed ‘hedges’ for ‘our health’ • e.g., Health Systems Evidence for ‘our system’  Staff a rapid-response unit (e.g., REACH Uganda, MHF)  Provide training (and tools) for decision-makers and stakeholders about how to find and use research evidence (e.g., MHF’s Health Systems Learning) Addressing Challenge 4b 12
  • 13. Challenge 4  Research evidence isn’t easy to use Challenge 4c  Decision-makers lack mechanisms to prompt them to use research evidence in decision-making One option (among many) for addressing challenge 4c  Propose changes to cabinet submissions and senior management committee briefings (and to periodic program reviews) to prompt analysts to summarize whether and how research evidence informed the clarification of a system problem, the framing of options to address the problem, and the proposed approach to implementation (e.g., Ontario’s Research Evidence Tool) Addressing Challenge 4c 13
  • 14. Challenge 4  Research evidence isn’t easy to use Challenge 4d  Decision-makers lack forums where challenges can be discussed with stakeholders and researchers One option (among many) for addressing challenge 4d  Plan deliberative dialogues at which pre-circulated evidence briefs serve as the starting point for off-the-record deliberations involving decision-makers, stakeholders and researchers (stakeholder dialogues) or exclusively citizens (citizen panels) (e.g., EVIPNet, MHF)... A big step beyond GOBSATT Addressing Challenge 4d 14
  • 15. A Potential Two-Pronged Approach Approach 1 – What researchers and ‘knowledge brokers’ can do Prepare evidence / citizen briefs and convene stakeholder dialogues / citizen panels over time frames of weeks and months Prepare rapid syntheses over time frames of days and weeks Promote one-stop shops and provide training so decision-makers and stakeholders can find research evidence on their own over time frames of hours and days Approach 2 – What decision-makers can do Send clear signals that using research evidence is a high priority Establish a performance requirement for staff Establish an internal rapid-response unit Establish a requirement to summarize whether and how research evidence informed understanding of the problem, options and implementation considerations before decisions are taken 15
  • 16. What Is Needed To Do This Work Knowledge (see the ‘summary sheet’) Questions to ask about a problem, options and implementation considerations Types of research evidence needed to answer these questions Appropriate sources of key types of research evidence What an AMSTAR score means Questions to ask about local applicability considerations Difference between knowing what other states or countries are doing (jurisdictional scans) versus knowing the results of research conducted in other countries (research evidence) 16
  • 17. What Is Needed To Do This Work (2) Attitudes Working iteratively to understand a problem, options and implementation considerations in light of institutional constraints, interest group pressure, values and many other types of information, as well as ‘external’ factors such as the state of the economy Being systematic and transparent in finding and using research evidence as one input to the decision-making process Finding and using the best available (i.e., highest quality, most locally applicable, synthesized) research evidence in the time you’ve been given Looking first for a perfect match in the available research evidence (to support an instrumental use) and then looking more broadly (to support a conceptual use) 17
  • 18. What Is Needed To Do This Work (3) Skills (for the ‘doers’) Clarifying a problem (and its causes), framing options to address the problem, and identifying implementation considerations Searching appropriate sources of research evidence Using AMSTAR to assess the quality of a systematic review Conducting a local applicability assessment Commissioning research to fill gaps in research evidence 18
  • 19. 1. What is the problem (and its causes)? 2. How did the problem come to attention and has this process influenced the prospect of it being addressed? 3. What indicators can be used, or collected, to establish the magnitude of the problem and to measure progress in addressing it? 4. What comparisons can be made to establish the magnitude of the problem and to measure progress in addressing it? 5. How can a problem be framed (or described) in a way that will motivate different groups? Clarifying the Problem - Questions 19
  • 20.  Low rates of childhood immunization  A risk factor, disease or condition – Re-emergence of some preventable childhood diseases, such as measles  A program, service or drug currently being used – Program coverage rate dropped 5% in the past year and the program has not corrected a celebrity’s assertions about safety and effectiveness  Current health system arrangements within which programs, services and drugs are provided • Governance arrangements – Fragmented accountability • Financial arrangements – Refugees have to pay out-of-pocket for immunizations for their children • Delivery arrangements – Many children are not registered with a primary care team that can proactively reach them  Current degree of implementation of an agreed course of action (e.g., a policy) – Some parents are not aware of new schedule Q1: What Is the Problem (and its Causes)? Example 1 20
  • 21.  Many citizens do not have a primary care physician  A risk factor, disease or condition – Rising rates of chronic diseases  A program, service or drug currently being used – Many citizens do not benefit from coordinated prevention and proactive management of chronic diseases  Current health system arrangements within which programs, services and drugs are provided • Governance arrangements – No mechanism to introduce foreign physicians without them ‘competing’ with local physicians • Financial arrangements – Most physicians are paid on a fee-for- service basis and can thrive even in well-served communities • Delivery arrangements – Many physicians work in solo practices  Current degree of implementation of an agreed course of action (e.g., a policy) – Policymakers do not enforce return-of-service agreements signed by physicians Q1: What Is the Problem (and its Causes)? Example 2 21
  • 22.  Problem clarification involves  Brainstorming about a problem and its potential causes (question 1)  Searching for relevant data (questions 2 and 3) and research evidence (questions 4 and 5) in appropriate sources (PubMed for single studies or Health Systems Evidence for systematic reviews related to health system arrangements) • See the summary sheet  Iteratively clarifying the problem (and its causes) in light of the data and research evidence found Clarifying the Problem - Summing Up 22
  • 23. 1. What is an appropriate set of options to address the problem? 2. What benefits are important to those who will be affected and which benefits are likely to be achieved with each option? 3. What harms are important to those who will be affected and which harms are likely to arise with each option? 4. What are the local costs of each option and is there local evidence about their cost-effectiveness? 5. What adaptations might be made to any given option and might they alter its benefits, harms and costs? 6. Which stakeholders’ views and experiences might influence the acceptability of an option and its benefits, harms, and costs? Framing Options - Questions 23
  • 24.  To address the problem of low rates of childhood immunization 1. Establish accountability among primary care practices for registering all children in their catchment area and for achieving a target immunization coverage rate (governance arrangement) 2. Remove all out-of-pocket charges for childhood immunization (financial arrangement) 3. Undertake a mass-media campaign to correct a celebrity’s assertions about the safety and effectiveness of childhood immunization (program change) and to raise awareness about a new immunization schedule (implementation strategy) Q1: What is an Appropriate Set of Options? Example 1 24
  • 25.  To address the problem of many citizens not having a primary care physician 1. Increase the supply of physicians by raising medical school enrolment (delivery arrangement) 2. Improve the distribution of physicians by enforcing return-of-service agreements signed by physicians (implementation strategy) 3. Improve the supply and distribution of physicians by introducing geographically restricted licences (governance arrangement) 4. Change the dominant physician-remuneration mechanism from fee- for-service payment to capitation (financial arrangement) Q1: What is an Appropriate Set of Options? Example 2 25
  • 26.  What benefits are important to those who will be affected and which benefits are likely to be achieved with each option? (Question 2)  Systematic reviews of effectiveness studies (e.g., randomized trials)  What harms are important to those who will be affected and which harms are likely to arise with each option? (Question 3)  Systematic reviews of effectiveness studies or observational studies  What are the local costs of each option and is there local evidence about their cost- effectiveness? (Question 4)  Local cost-effectiveness analysis  What adaptations might be made to any given option and might they alter its benefits, harms and costs? (Question 5)  Systematic reviews of qualitative studies (process evaluations)  Which stakeholders’ views and experiences might influence the acceptability of an option and its benefits, harms, and costs? (Question 6)  Systematic reviews of qualitative studies (e.g., acceptability studies) Research Evidence Can Help to Respond to the Additional Questions 26
  • 27.  Options framing involves  Brainstorming about an appropriate set of options to address a problem (question 1)  Search for research evidence (in the Cochrane Library + PubMed, Health Evidence + PubMed, or Health Systems Evidence) about each option in turn • See the summary sheet  Iteratively framing the options in light of the research evidence found Framing Options - Summing Up 27
  • 28. 28 Identifying Implementation Considerations - Questions to Consider 1. Where are the potential barriers to (and facilitators of) the successful implementation of an option? Are they at the level of o Patients/citizens (see question 2) o Health workers (see question 3) o Organizations (see question 4) o System (see question 5) 1. What strategies should be considered in order to facilitate the necessary behavioural changes among patients/citizens? 2. What strategies should be considered in order to facilitate the necessary behavioural changes among health workers? 3. What strategies should be considered in order to facilitate the necessary organizational changes? 4. What strategies should be considered in order to facilitate the necessary system changes?
  • 29.  Identifying implementation considerations involves  Brainstorming about (and searching for local data and research evidence about) potential barriers to (and facilitators of) the successful implementation of an option (question 1)  Searching Health Systems Evidence for research evidence about each potential strategy in turn • Benefits (question 2) • Harms (question 3) • Cost-effectiveness (question 4) • Adaptations (question 5) • Stakeholders’ views and experiences (question 6)  Iteratively calibrating the strategies in light of the research evidence found Identifying Implementation Considerations - Summing Up 29
  • 30. My Top Three Priorities for Action  Establish a requirement to summarize whether and how research evidence informed understanding of the problem, options and implementation considerations before decisions are taken  Promote training for (and the use of tools and resources by) decision- makers and stakeholders so they can find and use the best available data and research evidence – systematically and transparently – in the time available  Create incentives for and communicate priorities to those who have the knowledge, attitudes and skills to  Prepare rapid syntheses in days and weeks  Prepare evidence briefs (or citizen briefs) and convene stakeholder dialogues (or citizen panels) in weeks and months (and stop incentivizing researchers and university media offices to publicize the results of single studies without clear justification) 30
  • 31. Resources  Available on Health Systems Evidence (www.healthsystemsevidence.org)  Summary sheet on ‘finding and using research evidence’ (one page)  Hyperlinked list of SUPPORT tools (two pages)  Health Systems Evidence (four pages) 31