Using Data for Impact: A Critical ExaminationThrough the Lens of Implementation Science                     David Pelletie...
Outline1.   Why implementation science2.   Definition and state of the field3.   Broadening the field4.   A Consolidated F...
Why Implementation Science?                                     p2  •   Efficacy-to-Effectiveness  •   Knowledge-to-Action...
Why Implementation Science?                                                                  Fixsen et al., p74      U5 De...
Some Overall Observations from the Literature1. New: Implementation science is a new and still-emerging   field with littl...
What is Implementation Science?  Definition of Implementation (or Delivery) Science                        (from NIH Fogar...
Current State of the Field                             Fixsen et al., p74
Current State of the Field
Broadening the Definition1. Beyond “health care”  –   To community and prevention  –   To other sectors (ag, livelihoods, ...
Beyond “Evidence-Based Interventions”The Narrow View: Implementation refers to efforts to introduce  evidence-based interv...
Beyond “Scientific” KnowledgeDetached,                     Body ofHighly-                     Generalizable               ...
Opening the Black Box           Toward a Comprehensive Framework for                  Implementation Science              ...
A Consolidated Framework of Implementation      (factors operate at multiple levels, from provider to organization and cou...
Do We Need a Framework?  A Simple (Probably Painful) ExampleWhich of the following best describes the training approach  t...
Effectiveness of Training Methods:                              Results of a Meta-Analysis                                ...
A Consolidated Framework of Implementation      (factors operate at multiple levels, from provider to organization and cou...
An Example for Group Work:The 2010 WHO Guidelines on Infant   Feeding in the Context of HIV
The 2010 Guidelines   “The Challenges”
Policy Choice                            ARV+Breastfeeding                     No Breastfeeding  Basic                  a....
Group Instructions (30 mins)1. Each table has been assigned one of the following domains:   A. The intervention, innovatio...
Further work…–   Does the framework identify “hidden bottlenecks?”–   Are any factors missing from the framework?–   How c...
Thank You!
•   Title: Data for Impact: A Critical Examination Through the Lens of Implementation    Science•   Implementation science...
Assuming that “Breastfeed + ARVs” is the “National Policy”What factors will influence the counseling practices of health w...
Upcoming SlideShare
Loading in...5
×

Using data for impact_Pelletier_5.1.12

474

Published on

Published in: Health & Medicine, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
474
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
34
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The literature on implementation science has exploded in the last ten years or so but most of it is outside of nutrition. There now are some good frameworks to build upon, for studying implementation processes and for trying to improve them. For instance it now is clear that certain factors are important early in the implementation process and others are important later, and that there is an imbalance in what has been studied (adoption versus implementation).
  • The literature has clearly identified some approaches that do NOT work: guideline dissemination and training (by themselves), which is sobering considering how much we seem to rely on them.
  • It is useful to have a common framework in mind in prioritizing research topics – in order to get on the same page as soon as possible and in order to make informed judgments about priority topics. At the most general level many of the existing frameworks identify these three factors as being crucial: characteristics of the interventions, the new contexts and the implementation processes. The central notion is that an intervention that has been shown to be efficacious in a few contexts, via RCTs, may or may not be effective in new contexts. Much depends on how those contexts differ from the original and the quality of the implementation processes. I will elaborate on each of these in the following slides.
  • Bottom lines: The black box is quite complex.The interactions and importance of Interventions, Context and Processes are clear.There is an almost infinite numberof reductionistic studies that could be done to: a) document the influence of each of these alone or in combination; and b) to test various ways to change some of these factors for the better. But this would take decades and would not necessarily help us in the end because these factors are all part of an interactive system so we need to understand the system, not the parts in isolation.A more productive use of this framework is: a) to help us design implementation strategies with our eyes wide open (anticipating and addressing various barriers and enablers in advance); b) appreciate the importance of “Implementation Strategies and Processes” as THE pivotal factor for later outcomes; and c) provide some intermediate indicators that should be measured and monitored during implementation so that we can learn how they change under different circumstances.
  • Bottom lines: The black box is quite complex.The interactions and importance of Interventions, Context and Processes are clear.There is an almost infinite numberof reductionistic studies that could be done to: a) document the influence of each of these alone or in combination; and b) to test various ways to change some of these factors for the better. But this would take decades and would not necessarily help us in the end because these factors are all part of an interactive system so we need to understand the system, not the parts in isolation.A more productive use of this framework is: a) to help us design implementation strategies with our eyes wide open (anticipating and addressing various barriers and enablers in advance); b) appreciate the importance of “Implementation Strategies and Processes” as THE pivotal factor for later outcomes; and c) provide some intermediate indicators that should be measured and monitored during implementation so that we can learn how they change under different circumstances.
  • Using data for impact_Pelletier_5.1.12

    1. 1. Using Data for Impact: A Critical ExaminationThrough the Lens of Implementation Science David Pelletier Division of Nutritional Sciences Cornell University Spring CORE Group Meeting, Wilmington, DE May 1, 2012
    2. 2. Outline1. Why implementation science2. Definition and state of the field3. Broadening the field4. A Consolidated Framework5. An Example: Training frontline workers6. An Example: Infant feeding and HIV
    3. 3. Why Implementation Science? p2 • Efficacy-to-Effectiveness • Knowledge-to-Action • Evidence-Based Interventions • Evidence-Based Decisions • Evidence-Based Policy Making • Scaling Up • Etc.
    4. 4. Why Implementation Science? Fixsen et al., p74 U5 Deaths Preventable Through: Achieving 100% Coverage of Existing Interventions 62% Discovering and Delivering New Interventions 21% NIH/BMGF Research Funds (2000-04) Focused on: Improving delivery and utilization 3% Mechanistic and discovery research 97% Source: Leroy et al., AJPH 97(2), 2007
    5. 5. Some Overall Observations from the Literature1. New: Implementation science is a new and still-emerging field with little consistency in terms, concepts and methods2. Complex: There are ~40 categories of factors (and hundreds of variations on these) that affect the quality and impact of intervention delivery3. Contextual: The importance and changeability of these factors varies widely across countries, organizations and communities. Context is King.4. Conclusion: The focus of our research must shift from “technical interventions” to “implementation interventions”
    6. 6. What is Implementation Science? Definition of Implementation (or Delivery) Science (from NIH Fogarty International Center) “Implementation science is the study of methods to promote the integration of research findings and evidence into healthcare policy and practice. It seeks to understand the behavior of healthcare professionals and other stakeholders as a key variable in the sustainable uptake, adoption, and implementation of evidence-based interventions. […] The intent of implementation science and related research is to • investigate and address major bottlenecks , • test new approaches to prevent, detect and/or resolve bottlenecks • determine a causal relationship between the intervention and its impact” (at scale)
    7. 7. Current State of the Field Fixsen et al., p74
    8. 8. Current State of the Field
    9. 9. Broadening the Definition1. Beyond “health care” – To community and prevention – To other sectors (ag, livelihoods, social protection, etc)2. Beyond “evidence-based interventions” – To include IIGPP (next slide)3. Beyond “research findings and evidence” in the usual (privileged scientific) sense – To include practice-based and local knowledge and experience
    10. 10. Beyond “Evidence-Based Interventions”The Narrow View: Implementation refers to efforts to introduce evidence-based interventions into the practices of organizations and professionals (NIH, WHO, etc)A Broader View: Implementation refers to efforts to fulfill a goal or the intent of a policy, even when the evidence is lacking concerning how to do so (European Journal of Social Work 13(1):109, 2010)An Inclusive View: Implementation refers to efforts to introduce interventions, innovations, guidelines, practices or policies (IIGPP) into organizations, systems or communities
    11. 11. Beyond “Scientific” KnowledgeDetached, Body ofHighly- Generalizable FormalStructured or Principles/ M&EExperimental Scientific/Research Theoretical Knowledge IntegrationReflection, Application ofLearning, Knowledge in ImprovedDocumentation, Program Design ActionCodification & Implementation Contextual Knowledge SystematizedPractice-Based & Tacit Experience &Experience & Knowledge FeedbackEngaged DuringResearch Implementation
    12. 12. Opening the Black Box Toward a Comprehensive Framework for Implementation Science New Intervention Contexts Proven in OutcomesA Few Contexts At-Scale (RCTs) Dissemination & Implementation Processes
    13. 13. A Consolidated Framework of Implementation (factors operate at multiple levels, from provider to organization and country) Context D. Outer Setting: Participant needs & resources, govt and donor policies & regulations, peer/ network influences , societal & cultural influences, etc.A. Intervention/ Implementing OrganizationInnovation / E.Guideline/ Implementation B. Inner Setting: orgPractice / size, structure, maturity, manage Processes:Policy (IIGPP) •Formative research(unadapted) ment, informal Implementation networks, culture, climate •Planning•Core components •Implem Strategy Outcomes•Peripheral (tension, compatibility, priority, in centives, goal clarity, learning •Change Theory Acceptabilitycomponents Service climate), readiness •Engaging 1. Fit Client Outcomes (leadership, resources, access to -opinion leaders 2. Feasibility OutcomesPerceived and knowledge & info) -formal leaders 3. Costs 1. Efficiency -HealthActual:source, IIGPP (adapted) -champions 2. Effectiveness - Function Applicationevidence, advant •Core components -facilitators 3. Equity -Symptomsage, adaptability, • Execution 1. Adoption 4. Safety -Satisfactiontrialability, compl •Peripheral -components 2. Adaptation 5. Patient-exity, design components -sequence 3. Fidelity/Quality centeredquality and -intensity 4. Penetration 6. Timelinesspackaging, cost C. Individuals (providers -duration 5. Sustainability and managers): knowledge -quality & beliefs, self-efficacy, stage of •Feedback change, identification with •Evaluation organization, motivation, values, •Reflection Adapted from: •Decisions Damschroeder et al., Implem Science 4:50, 2009; Proctor et al., Admin Pol Mental Hlth 38:65-76, 2011 intellect, competence, capacity, l •Adjustments
    14. 14. Do We Need a Framework? A Simple (Probably Painful) ExampleWhich of the following best describes the training approach typically used with frontline workers in your programs?1. Lectures and Discussion2. Lectures and Discussion + demonstrations3. Lectures and Discussion + demos + hands-on practice and feedback
    15. 15. Effectiveness of Training Methods: Results of a Meta-Analysis Outcomes for Assessing Effectiveness Teacher’s Teacher’s Teacher’s Training Knowledge Skill Use of New Method Practices Lectures and 10% 5% 0 Discussion Why? + Demonstrations 30% 20% 0 + Practice and 60% 60% 5 Feedback + Coaching in the 95% 95% 95% ClassroomFixsen, D.L., et al., Implementation research: A synthesis of the literature. 2005,National Implementation Research Network: Tampa, FL.
    16. 16. A Consolidated Framework of Implementation (factors operate at multiple levels, from provider to organization and country) Context D. Outer Setting: Participant needs & resources, govt and donor policies & regulations, peer/ network influences , societal & cultural influences, etc.A. Intervention/ Implementing OrganizationInnovation / E.Guideline/ Implementation B. Inner Setting: orgPractice / size, structure, maturity, manage Processes:Policy (IIGPP) •Formative research(unadapted) ment, informal Implementation networks, culture, climate •Planning•Core components •Implem Strategy Outcomes•Peripheral (tension, compatibility, priority, in centives, goal clarity, learning •Change Theory Acceptabilitycomponents Service climate), readiness •Engaging 1. Fit Client Outcomes (leadership, resources, access to -opinion leaders 2. Feasibility OutcomesPerceived and knowledge & info) -formal leaders 3. Costs 1. Efficiency -HealthActual:source, IIGPP (adapted) -champions 2. Effectiveness - Function Applicationevidence, advant •Core components -facilitators 3. Equity -Symptomsage, adaptability, • Execution 1. Adoption 4. Safety -Satisfactiontrialability, compl •Peripheral -components 2. Adaptation 5. Patient-exity, design components -sequence 3. Fidelity/Quality centeredquality and -intensity 4. Penetration 6. Timelinesspackaging, cost C. Individuals (providers -duration 5. Sustainability and managers): knowledge -quality & beliefs, self-efficacy, stage of •Feedback change, identification with •Evaluation organization, motivation, values, •Reflection Adapted from: •Decisions Damschroeder et al., Implem Science 4:50, 2009; Proctor et al., Admin Pol Mental Hlth 38:65-76, 2011 intellect, competence, capacity, l •Adjustments
    17. 17. An Example for Group Work:The 2010 WHO Guidelines on Infant Feeding in the Context of HIV
    18. 18. The 2010 Guidelines “The Challenges”
    19. 19. Policy Choice ARV+Breastfeeding No Breastfeeding Basic a. Widespread HIV testing a. Safe water and sanitation are assured Requirements b. Reliable ARV supplies at household and community level if child survival is c. Full ARV adherence b. Sufficient infant formula is assured to be maximized d. Proper EBF adherence c. Can prepare it cleanly and frequently and transition to mixed d. Exclusive formula for first six months is feeding possible e. Family is supportive of this practice f. Access to health care that offers comprehensive child health services•What factors in the framework will affect the policy choice of national policy makers?•What factors in the framework will affect the policy choice of regional and district managers?•What factors in the framework will affect the counseling practices of front-line staff? Note: Factors affecting practices of mothers not included here
    20. 20. Group Instructions (30 mins)1. Each table has been assigned one of the following domains: A. The intervention, innovation, guideline, practice or policy (IIGPP) B. Individuals (providers and managers) C. The Inner Setting (the implementing organization) D. The Outer Setting (actors and conditions outside the organization that affect implementation) E. Implementation processes (activities and practices for “rolling out” the guideline)2. Identify some of the characteristics of your domain that may positively or negatively affect the implementation of the guideline (“implementation” here refers to adoption, adaptation, penetration to all clinics, and quality of delivery)3. How would you collect data or information before implementation in order to plan your implementation strategy?4. Chose one example to report out.
    21. 21. Further work…– Does the framework identify “hidden bottlenecks?”– Are any factors missing from the framework?– How can the framework be made more user-friendly?– How can we be innovative in assessing and re-assessing the factors?– Who needs to be assessing and acting on various ones?
    22. 22. Thank You!
    23. 23. • Title: Data for Impact: A Critical Examination Through the Lens of Implementation Science• Implementation science seeks to improve the effectiveness of large-scale programs by strengthening the frameworks, tools and evidence base for identifying and minimizing implementation bottlenecks. Implementers can make major contributions to this field because of their extensive knowledge, experience and influence in the implementation process. This session provides an overview of this emerging field and introduces one of the many conceptual frameworks in the literature for guiding the assessment and improvement of the implementation process. It also seeks participants’ views on the forms and meaning of “data” in the context of real-world implementation by engaging participants in a rapid application of this framework, using as a case study the WHO Guidelines on infant feeding in the context of HIV.
    24. 24. Assuming that “Breastfeed + ARVs” is the “National Policy”What factors will influence the counseling practices of health workers and the feeding practices of mothers? CFIR Domain CFIR Element Characteristics of the intervention Perceived strength of evidence Perceived advantage Complexity Characteristics of the individuals Knowledge and beliefs Identification with the organization Characteristics of the inner setting Goal clarity Compatibility Characteristics of the outer setting Perceived social, cultural and economic situation of the mother/caretaker Donor influences on MOH policy Characteristics of the Implementation strategy (train and hope?) implementation process Evaluation Adjustments
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×