Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Faster Improvement with Adaptive Implementation Research

295 views

Published on

Feb 3, 2016
Dr. John Ovretveit, Director of Research and Professor of Health Innovation and Evaluation at the Karolinska Institutet, presented as part of a seminar series on UCLA CTSI Dissemination, Improvement and Implementation Research.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Faster Improvement with Adaptive Implementation Research

  1. 1. Faster Improvement with Adaptive Implementation Research 1 John Øvretveit, Director of Research, Professor of Health Innovation and Evaluation, Karolinska Institutet, Stockholm, Sweden 2/3/2016
  2. 2. Help with these challenges  Changing clinical practice and service delivery  eg use this test in these situations not these, establish team-based service  Effective methods for change under-used  by improvement projects  Frustration of practical improvers  research not answering their questions  how do we implement this? costs and savings? conditions we need to get the same results?  Researcher’s challenges  reducing time and cost;  Time and funding to publish  Role for working on implementation – for trial or spread – time and funding  documenting changes;  attributing outcomes (esp which part most impactful);  generalising; take-up of our research 22/3/2016
  3. 3. Does not help with …  Changing academic research criteria  Not improver’s questions  Do not reward “3R research” relevant, responsive, rigorous  Rather “3P review criteria” for proposals, publications and promotion, using RCT, ideally  Settings where implementation is not possible  but will know why 3
  4. 4. Relevance to researchers ..if evaluating an intervention  (e.g. new treatment or service delivery model) .. knowledge of non-experimental methods  which data to gather to document and evaluate ..designing effective intervention (eg protocol for establishing an intervention to test)  Issues in translating research into practice and scale-up – researcher’s role? 42/3/2016
  5. 5. It was effective there… but different here…? 52/3/2016 We don’t have a problem here Our patients/service is different We have other changes and priorities
  6. 6. Session covers  Case example - problems  Proven solutions – why not implemented?  “Fit” of Solution in the “Context”  Does the adaption work? Adaptive implementation research  Research to answer improver’s questions  Implications for you 62/3/2016
  7. 7. What I mean by… “Intervention” covers both a)Do hand hygiene consistently (before/after change) b) Training and feedback (actions to enable change) Words a) the “new better way” - hand hygiene  eg wash hands between patients; use bar code reader to reduce medication errors; rapid response team to prevent avoidable deterioration on nursing units. (The Improvement-change) b) What we do directly to enable “take up” by staff of the new better way (“implementation” methods - training, feedback, rewards, punishment for not using, etc.) 72/3/2016
  8. 8. . 82/3/2016 B Better hand hygiene A Poor hand hygiene Intervention Plan Wash hands between patients Reduce barriers Feedback Implementation actions (phases) 1Education 2Gell dispensers 3Patient expectations 4Feedback compliance (infection rates) How? Surrounding “context” helps and hinders Intervention concept Wash hands between patients Reduce barriers Feedback
  9. 9. What I mean by…  Conditions or context for the intervention: = influences which indirectly help or hinder the intervention  high workload affects hand hygiene,  disruption to supplies,  IT system,  how the hospital or physicians are paid 92/3/2016
  10. 10. 2/3/2016 10 Mary: 84 yrs. Obstructive airways (COPD), heart disease, mild depression Stable at home on meds, very independent Unpaid motivational coach and security-guard - “Matty”
  11. 11. Healthcare experience  Emergency hospital admission  Delays, and meds changed  HAI  Sent home – no comms or support  Readmission  Before/after 1 12/3/2016
  12. 12. Mary - six weeks later  After hospitalisation  Avoidable cost 4600$ 1 22/3/2016
  13. 13. Name a proven interventions for any problems? Changes of meds/wrong meds? Hospital acquired infection? After care transition?  Medication reconcilliation  Hand hygiene - bundle of interventions  After-care - Coleman CTI – Transition coach and education 1 32/3/2016
  14. 14. Your experience? We have implemented solutions to prevent this - Yes, No, sometimes  We have effective Med Rec - Medications changes unlikely  HAI from Hand Hygiene non-compliance unlikely  After care information to PCP ensured within 2 days of discharge  Effective after-care support for older multiple morbidity patients 1 42/3/2016
  15. 15. Why not implemented? - your vote  No problem (one-off event)  Not aware of these proven effective solutions  Know how to implement - don’t have time or resources to implement  To implement effectively - don’t have knowledge and skills  Know solutions - uncertain if we would get same results - would it work here?  Don’t have research to show how to implement in different settings 1 5
  16. 16. Johns Quest…  Why not implemented?  What would help fast & widespread implementation?  How can research best help? 1 62/3/2016 Conclusions so far… …in 2 slides
  17. 17. 2/3/2016 17 Success depends on… Seed Gardener/planting & nurture Soil / climate Idea 10% Adaption/Implementation 30% Personalities 20% The 10;20;30;40 change success rule Soil receptive – staff readiness & wider Climate 40%
  18. 18. No intervention survives first contact with context  . 1 82/3/2016 Implemented as planned? Intervention plan
  19. 19. Adapt to survive  Adapt the improvement change tested elsewhere  E.g. turn at risk patients every 4hrs not 2hrs (Pressure ulcer intervention)  Adapt the context  Increase staffing by 0.5 FTE  Both - Adapt the change & context  Need “Goldilocks Improvement-Fit, for Take Up” GIFTU 1 9
  20. 20. Context – innovation “Fit” 2 02/3/2016
  21. 21. Fidelity and adaptive implementation  .  t 2 12/3/2016 Adapt to “Fit”, to setting and subjects Adjust over time – dynamic>>>>> To do this-Resources, data and skills
  22. 22. Research questions…  Is the adaptation more, or less effective?  How can we find effective adaptations in different settings?  How can we do lower-cost faster research  on questions which will help take up 2 22/3/2016
  23. 23. Research “Case” example Improving physicians hand hygiene compliance – most effective intervention? Pilot study, 8 hospitals using QI methods Identified 24 causes. Different in each hospital Interventions tailored to local cause 2nd study – web based tool Collect data about your causes (from 24) and used these interventions to fix your local causes  8 Pilots compliance 48% / 81% ;  Vs 174 organisations 58% / 84% (769 2 3
  24. 24. Variation reported in a UK study  . 2 42/3/2016
  25. 25. One lesson for research from case?  Providers supplied data  Method enables easy local adaptation  But adaptations made are unknown  Effectiveness on average  not the only knowledge needed by implementers  Need data to explain variations  Not implemented  Context hinders  Adapt successfully or unsuccessfully 2 52/3/2016
  26. 26. More informative research Experimental Trials  Can and should we explain high and low outcomes/ performers?  Can we do subgroup analysis?  Focus on non-experimental observational studies 2 62/3/2016
  27. 27. Do you have any of these “Research challenges”? Documenting?  Use taxonomies of implementation methods  Use adaptation typology  Attributing?  Use Implementation outcome model  Use Logic Model/programme theory  Generalising? – guidance  Find which adaptations in which context get which results  Use? (take up by practitioners) = need Pre-study planning, to get data for, 2 7
  28. 28. Exercise You are the head of a clinic:  To enable physicians to follow antibiotic prescribing guidelines…  Which behavioral change techniques would be most effective?  Would you use the same in FQC PHC and Regan paediatrics outpatients? 2 82/3/2016
  29. 29. Selection from 93 behavior change techniques (Michie2013) 1. Instruction on how to perform a behavior 2. Punishment 3. Anticipation of future rewards or removal of punishment 4. Practice with Graded tasks 5. Mental rehearsal of successful performance 6. Examine Anticipated regret 7. Feedback on behavior 8. Other(s) monitoring with awareness 9. Social comparison 10. Behavioral contract 11. Social support 12. Identification of self as role model 2 92/3/2016
  30. 30. See also Powell 2015: 73 implementation strategies Mixes direct interventions and indirect context changes:  Education  Audit and provide feedback  Create or change credentialing and/or licensure standards  Develop disincentives  Use capitated payments Do we need to distinguish? 3 02/3/2016
  31. 31. Document Adaptions 1Who made the modification? 2What was modified? (Stirman 2009) 3 12/3/2016
  32. 32. Documenting critical context factors - framworksCFIR (Damschroder 2009) PHARIS (Rycroft-Malone 2002) ORCA readiness - based on PHARIS MUSIQ (Kaplan et al 2010) - QI French et al 2009 (review of context measures for evidence-based practice (EBP)) 3 22/3/2016
  33. 33. Recommended practical tools ORCA (or CFIR http://www.cfirguide.org/) 3 3
  34. 34. change readiness (HRET) 4 areas: the innovation, target audience, the organisation, the environment, 3 4
  35. 35. Attributing – use causal chain frameworks Theory informed Case evaluation  .  .  . 3 52/3/2016
  36. 36. Programme theory models, or logic models 3 62/3/2016
  37. 37. Attributing – use causal chain frameworks 3 72/3/2016
  38. 38. Attributing – check implementation, before later outcomes(Proctor 2012) 3 82/3/2016 Johns quest – enlightenment!
  39. 39. Summary Challenges both implementing, and researching implementation One approach = Adaptive implementation research Plan data to gather to  Document adaptation and context  Assess implementation outcomes  Attribute outcomes through causal chain map 3 92/3/2016
  40. 40. Implications for you  Trial - Experimental evaluation – who doing this?  Do we need to identify and explain “outliers”?  Do we have data to do this?  Non-experimental/observational - – who ? (inc. natural experiments) - ob  Plan and use existing data  Ask a cross-section of “informed observers”  Find out implementers needs for information effectively to take up improvements  Responsible for implementation during or 4 02/3/2016
  41. 41. Implications for you  Responsibilities for implementation, during or after your research? “Another essential component of implementation research involves the enhancement of readiness through the creation of effective climate and culture in an organization or community” (Implementation research glossary, Brownson et al 2012) 4 12/3/2016
  42. 42. Questions to you  Do you study implementation?  What do you need to do better in your implementation research?  Anything a surprise?  Examples or experience you have about this? 4 22/3/2016
  43. 43. Questions posed  Why not implemented?  What would help fast & widespread implementation?  How can research best help?  how best can implementation researchers help practitioners to implement proven improvements locally?  Is it really their role to do so? 4 32/3/2016
  44. 44. .  .  .  . 4 42/3/2016
  45. 45. What do you think? Anything a surprise? Examples or experience you have about this? 4 52/3/2016
  46. 46. 6)  . 4 62/3/2016
  47. 47. .  .  .  . 4 72/3/2016
  48. 48. .  .  .  . 4 82/3/2016
  49. 49. What do you think? Anything a surprise? Examples or experience you have about this? 4 92/3/2016
  50. 50. . Conclusions through Discussion 5 02/3/2016
  51. 51. Which was most surprising,interesting or useful – vote 1) . 5 1
  52. 52. . Surprises? Most useful? Might not be true for us? 5 22/3/2016
  53. 53. Action:Pre-mortem Crystal ball shows 1 year ahead no results from the project 1)What are the most likely causes? 2)How could we have known before? 3)Possible actions we can take now? 5 3
  54. 54. Resources 5 42/3/2016
  55. 55. Frameworks for studying context – list (pictures later) See references  Clinicians implementing clinical research:  PARHIS Guidance (Stetler et al 2011) and related:  Context Assessment Index (CAI) (McCormack et al 2008) – the best validated  Alberta Context Tool (ACT) (Estabrooks et al 2008).  CFIR Damschroder et al 2009, and French et al 2009  Quality Improvement Projects : MUSIQ (Kaplan et al 2012) or French et al 2009)  See also ORCA readiness for change assessment (Helfrich et al 2009). 5 52/3/2016
  56. 56. BUT, how many services can do this… “evaluating the program and continually adapting the program on the basis of evaluation results and changes to the context of your organization (e.g., changes in staff, changes to the community, changes in the population served) can help ensure that the program remains relevant and addresses any potential challenges that occur over time” “Evaluation”? = lunchtime review <<<>>>RCT 5 6
  57. 57. Strategy Steps over time Feb 1)Form project team March 2)Gather initial data April 3) etc. 57 Supports  Systems for data  Facilitators 2/3/2016 “3S framework” to describe “Implementation approach Structure responsibilities; accountability reporting Example – QI breakthrough collaborative
  58. 58. Recommended practical tools Brach et al 2008: Will it work here? 5 8
  59. 59. French et al 2009 synthesis of 30 instruments measuring context 5 9
  60. 60. Context for EBP (PARiHS) Rycroft  .C 6 0
  61. 61. 2/3/2016 61 Barriers analysis – the beginning of “context discussion”
  62. 62. Example 1 – adoption by clinicians of recommendation for more effective treatment of depression Medications & CBT for moderate depression.  Implementation strategy: Simple guidelines, training, feedback on outcomes, access to experts.  Study found  patient outcomes 10% improved.  Guess % physicians following guidelines?  40% not following guidelines  “Implementation fidelity not met”  practice changed in some but not fully, and in many, not at all  Phase 2 found barriers to change  consultation time extra, and opinion leader “lukewarm” 6 22/3/2016
  63. 63. 2)Barriers to take up  the lack of information technology systems;  physician culture,  beliefs and habits;  development & function of guideline  (Kenefick, et al 2008) 6 32/3/2016
  64. 64. Example studies of implementation strategies Specialist-nurse led clinics for implementing lipid control cost- effective (Mason 2005) Multifaceted strategy for CBT cost-effective (Scheeres 2008 & gen Mortimer 2013) Financial incentives for implementing ACE inhibitor & other Qndic cost-effective (Walker 2009) Audit and feedback for implementing intensified control of blood glucose is cost-effective (Hoomans 2009) No co-payments for implementing preventive medication is cost- effective (Choudry 2011) Structured patient education with group for implementing self- management is not cost-effective (Gillespie 2014) 6 4
  65. 65. Types of fidelity 1Copy the proven intervention  Treatment, practice, service delivery model  Whatever it takes to reproduce this in every day life and operations 2Copy the implementation approach  To enable patients to take the intevention, use exacty the same reminder system they found was effective for enabling patient uptake 3Copy both 4Copy the logic of the intervention 65 The letter kills but the spirit gives life
  66. 66. 4)Copy the logic of the intervention – the spirit The effective ingredients to enable practitioners to follow hand hygiene were:  Motivation (e.g. patient talks about MRSA)  Ability (Gell dispensers everywhere, agreement excuse for late/take longer)  Triggers (reminders)  Rewards (performance feedback, etc.) You make the mix which fits your service Is that adaption or fidelity to logic or both? 6 6
  67. 67. 5)Fidelity to guidance for adaption for targeting or tailoring  Following the guidelines for adapting treatments to older patients with multiple morbidity 6 7
  68. 68. Resources Tools Web sites 6 82/3/2016
  69. 69. References 6 92/3/2016
  70. 70. References 7 02/3/2016
  71. 71. DETAILS 7 12/3/2016
  72. 72. Practice-customer’s questions: Is the implementation approach effective for enabling providers to take up the new way? In setting where we can rigorously evaluate it? In a range of typical settings Does the “new way” then result in better patient and cost outcomes? What conditions do we need to implement it? Which adaptations are possible? How much does it cost and how long before results?7 2
  73. 73. 2) Action evaluation  Researchers share tasks with implementers in all stages of research  Researchers present their data to implementers  Implementers adjust intervention (or not)  +ve gain insights and data  -ve explain effects of researchers (additional intervention)  -ve researchers not available in other typical settings (generalisation) 7 32/3/2016
  74. 74. Coleman care transitions model = people leaving hospital - support for self care 7 42/3/2016 1)Education 2)Coach support at home
  75. 75. RCT evaluated – proven effective  Research funded version  Intervention specified in protocol  Implementation not described 7 52/3/2016
  76. 76. Groups on Coleman dissemination evaluation 7 6
  77. 77. Evaluation type 1: Fidelity to planned practice  Coleman 2013 core elements to get outcomes 7 72/3/2016
  78. 78. Groups on Coleman dissemination evaluation Did this study evaluate implementation?  What data did they collect? How?  What was “the intervention” and what was “the implementation” activities, structure and support?  How do we know if their conclusions follow from the data or a personal opinion? 7 8
  79. 79. Did subjects “take up” the change as intended? Evaluation type 2: for adaptive implementation How well did they adapt intervention during implementation better to achieve implementation outcomes Role of evaluation = a) Help adaption, b) feedback on success 7 9
  80. 80. Other studies Descriptive study, by facilitators, of adaptions …with unclear evidence of outcomes, but very useful to practice improvement - More funding for and research of this type? 8 02/3/2016
  81. 81. Partnership research with SLL 8 1 Improvement projects
  82. 82. John Øvretveit – Medical Management Center Karolinska Institutet Applied healthcare research in a medical university Grown since 2002 to 87 staff and 35 PHD researchers Example projects Funding models for Value based purchasing Implementation of improvements for chronic care Integrated care Most mixed methods, many non-experimental Details: http://ki.se/en/lime/medical-management-centre 8 22/3/2016
  83. 83. Culture and translation . 8 32/3/2016 The strengths of Ca and US § diversity, § innovation, § entrepreneurship, § openness, § can come together rapidly and attack a problem, § massive resources and talent How are you going to create into the future the new California public health? 21/3/16 “No pants metro day”
  84. 84. 8 4
  85. 85. .  . 8 52/3/2016
  86. 86. 2/3/2016 86 Conclusions 1. Surprises… 2. Useful… 3. Not mentioned …Look this up…

×