Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Implementation Research:
What is it? Am I doing it?
How can I do it better?
Olakunle Alonge, Assistant Scientist,
Health S...
Learning Objectives
By the end of this session, participants will be able to:
• Explain what Implementation Research (IR) ...
Implementation Practice and Research
• “Implementation [practice] is the act of carrying an
intention into effect, which i...
Implementation Research
Translational Research
Implementation Research and Other
Common Studies in Public Health
4
T4
stud...
IMPLEMENTATION OUTCOMES:
FRAMEWORK AND DEFINITIONS
5
Implementation Outcomes
• Implementation variables:
– Constructs whose vector properties (magnitude and direction)
changes...
Conceptual Framework for Implementation
Outcomes
Clients Outcome
Satisfaction
Symptomatology
Function
Population-
Based
In...
Implementation Outcomes - Definitions
Implementation
Outcome
Working Definition* Related terms**
Acceptability The percept...
Implementation Outcomes - Definitions
Implementation
Outcome
Working Definition* Related terms**
Fidelity The degree to wh...
Measuring Implementation Outcomes
10 Source: Proctor et al 2011
Measuring Implementation Outcomes
11 Source: Proctor et al 2011
FIDELITY
12
Fidelity - Definition
• Degree to which an intervention was implemented as
prescribed by the original protocol or program ...
Fidelity – Domains of Interest
• May consist of the following domains:
14 Source: Dane & Schneider, 1998; Dusenbury et al ...
Fidelity – Domains of Interest
• Domains of interest:
– Adherence – an intervention is being delivered as it was designed ...
ACCEPTABILITY
16
Acceptability
• Perception among stakeholders (e.g. consumers, providers,
managers, policy-makers) that an intervention is...
FEASIBILITY
18
Feasibility
• The extent to which an intervention can be carried out in a
particular setting or organization
– Typically a...
Feasibility – Domains of Interest
20 Source: Bowen et al. AJPM 2009
Feasibility – Domains of Interest
21 Source: Bowen et al. AJPM 2009
Summary – What is IR? / IR Outcomes
• Implementation research is the scientific inquiry into questions
concerning implemen...
FORMULATING IMPLEMENTATION RESEARCH
PROBLEMS AND QUESTIONS
23
24
Addressing Public Health Problems
Define the problem Measure the magnitude of the problem
Develop a conceptual framewor...
25
Problem Definition
Creating a problem definition is an iterative
process
• Non-linear process
• Changes in the definiti...
Prioritization of Health Issues for IR
• Priority Rating Formula (Hanlon)
– Component A = Size of the problem
– Component ...
27
Specificity of the Problem
• What is the specific dimension or aspect of interest?
• If you think you are being too spe...
Key Aspects of a Problem Definition
• Nature of the problem/focus
• Framing of the problem
– Person
– Place
– Time
• Speci...
29
Framing of the Problem
Person
• Who is affected by this problem?
• Be as specific as possible…
– …using basic demograph...
Place
• Where is the problem located?
• Be as specific as possible:
– Geographical
– Cultural
– Social
– Political
– Econo...
Time
• What is the specific time frame for the problem?
• Be as specific as possible
– Dates (1990-2000)
– Time since a sp...
Summary Problem Statement
• Summarizes the purpose of the study
• Establishes the direction and captures the essence
of th...
IR Questions
• Of interest to stakeholders
– Government, community, academia, implementing partners
• Answerable
• Shaped ...
34
Language Considerations
• Be clear and concise
• Define and reference technical terms
• Define local terms
• Generally ...
35
Causal Claims
• Are etiologic factors or causes part of your problem
statement?
• AVOID causal claims
• Be short and cl...
Small Group Assignment
• Statement of the problem for your IR proposal
– 1-2 paragraphs
• Formulate one specific IR object...
Upcoming SlideShare
Loading in …5
×

Implementation Research-What is it?

702 views

Published on

Implementation Research-What is it?

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Implementation Research-What is it?

  1. 1. Implementation Research: What is it? Am I doing it? How can I do it better? Olakunle Alonge, Assistant Scientist, Health Systems Program, JHBSPH Jim Ricca, MCSP Learning and Implementation Science Team Leader Stefan Baral, JHBSPH
  2. 2. Learning Objectives By the end of this session, participants will be able to: • Explain what Implementation Research (IR) is and how it related to other forms of research • Explain three commonly studied Implementation Research outcomes • Design answerable Implementation Research questions of importance to key stakeholders 2
  3. 3. Implementation Practice and Research • “Implementation [practice] is the act of carrying an intention into effect, which in health can be policies, programmes, or individual practices (collectively called interventions)” e.g. the act of administering misoprostol for AMTSL • “Implementation research is the scientific inquiry into questions concerning implementation” e.g. study on the acceptability of misoprostol among pregnant women in an area 3 Source: Peters, Adam, Alonge, Agyepong and Nhan, BMJ 2013; Gilbert, JRScT 1991
  4. 4. Implementation Research Translational Research Implementation Research and Other Common Studies in Public Health 4 T4 studies T3 (D&I) studies Intervention studies T2 studies T1 studies QI studies Operation Research M&E Source: Lobb and Colditz, Annual Rev Pub Health 2013; IRDS Consultative Meeting Report, 2014
  5. 5. IMPLEMENTATION OUTCOMES: FRAMEWORK AND DEFINITIONS 5
  6. 6. Implementation Outcomes • Implementation variables: – Constructs whose vector properties (magnitude and direction) changes with an implementation process • Implementation outcomes: – Implementation variables that describe effects (results) of a deliberate and purposive action to implement an intervention – Useful for measuring success or failure of an implementation process – Intermediate outcomes of the effectiveness of an intervention (on individual/population health outcomes and quality of life) – Underdeveloped constructs; operates across socio-ecological levels 6 Source: Peters, Adams, Alonge et al 2013; Proctol et al. 2011; Fixsen et al. 2005
  7. 7. Conceptual Framework for Implementation Outcomes Clients Outcome Satisfaction Symptomatology Function Population- Based Incidence of diseases Morbidity Mortality DALYs Health Outcomes Efficiency Coverage Equity Responsiveness Services Outcomes** Acceptability Adoption Appropriateness Costs Feasibility Fidelity Penetration Sustainability Implementation Outcomes I* 7 • *measurable both at the individual and aggregated level • **measurable mostly at an aggregated level Source: Proctor et al 2011
  8. 8. Implementation Outcomes - Definitions Implementation Outcome Working Definition* Related terms** Acceptability The perception among stakeholders (e.g. consumers, providers, managers, policy-makers) that an intervention is agreeable Related factors: (e.g. Comfort, Relative advantage, Credibility) Adoption The intention, initial decision, or action to try to employ a new intervention Uptake, Utilization, Intention to try, Appropriateness The perceived fit or relevance of the intervention in a particular setting or for a particular target audience (e.g. provider or consumer) or issue Relevance, Perceived fit, Compatibility, Perceived usefulness or suitability Feasibility The extent to which an intervention can be carried out in a particular setting or organization Practicality, Actual fit, Utility, Trialability 8 Source: Proctor et al 2011; Peters, Adams, Alonge et al 2013
  9. 9. Implementation Outcomes - Definitions Implementation Outcome Working Definition* Related terms** Fidelity The degree to which an intervention was implemented as it was designed in an original protocol, plan, or policy Adherence, Delivery as intended, Integrity, Quality of delivery, Intensity or dosage of delivery Implementation cost The incremental cost of the implementation strategy (i.e. how the services are delivered in a particular setting); including the cost of the intervention itself. Marginal cost, Total cost*** Penetration/ Coverage Extent to which a practice or program is integrated within a service settings and system (niche saturation). The degree to which the population that is eligible to benefit from an intervention actually receives it. Reach, Access, Service Spread or Effective Coverage, Penetration Sustainability The extent to which an intervention is maintained or institutionalized in a given setting. Maintenance, Continuation, Routinization, Institutionalization, 9 Source: Proctor et al 2011; Peters, Adams, Alonge et al 2013
  10. 10. Measuring Implementation Outcomes 10 Source: Proctor et al 2011
  11. 11. Measuring Implementation Outcomes 11 Source: Proctor et al 2011
  12. 12. FIDELITY 12
  13. 13. Fidelity - Definition • Degree to which an intervention was implemented as prescribed by the original protocol or program developers • Adherence to the core program components – Core components: elements without which the intervention will not have its intended effect – Balancing fidelity and adaptation: 100% fidelity may be unrealistic 13 Source: Dusenbury et al 2003, Rabin et al. 2008
  14. 14. Fidelity – Domains of Interest • May consist of the following domains: 14 Source: Dane & Schneider, 1998; Dusenbury et al 2003; Carroll et al. 2007; Durlak and Dupre 2008; Proctor et al 2011 Dane & Schneider, 1998 Carroll et al. 2007 Proctor et al. 2011 Durlak and DuPre 2008 Adherence Adherence Adherence Adherence Dose Exposure or dose Dose Quality of delivery Quality of delivery Quality of delivery Participant responsiveness Participant responsiveness Program differentiation Program differentiation Intervention complexity Facilitation strategies
  15. 15. Fidelity – Domains of Interest • Domains of interest: – Adherence – an intervention is being delivered as it was designed or written – Dosage or exposure - frequency and duration of the intervention is as full as prescribed by its designers. Could mean amount of intervention that the recipient received – Quality of delivery - the manner in which an intervention worker (e.g. teacher, volunteer, or staff member) delivers a program • Based on techniques prescribed by developer • Based on a theoretical ideal – Participant responsiveness - how far participants respond to, or are engaged by, an intervention – Program differentiation - identifying unique features of different components of an intervention, and identifying the core components • Perspectives in measurement: – Representative (commonest approach) – Comprehensive – Complex 15 Source: Mihalic 2004, Carroll 2007
  16. 16. ACCEPTABILITY 16
  17. 17. Acceptability • Perception among stakeholders (e.g. consumers, providers, managers, policy-makers) that an intervention is agreeable – Based on direct knowledge of or experience with specific aspects of the intervention, e.g. content, comprehensiveness, ease of use, comfort, etc. – Distinguished from service satisfaction which often focuses on general service experience – Dynamic and changes with experience • Often conceptualized as consisting a single domain 17
  18. 18. FEASIBILITY 18
  19. 19. Feasibility • The extent to which an intervention can be carried out in a particular setting or organization – Typically assessed retrospectively to explain success or failure of an implementation process (actual fit) – Closely related to appropriateness which looks at the perceived fit of an intervention for a setting (i.e. prospectively assessed) • Has multiple domains and similar measurement perspectives as in fidelity (i.e. representative, comprehensive, complex) 19
  20. 20. Feasibility – Domains of Interest 20 Source: Bowen et al. AJPM 2009
  21. 21. Feasibility – Domains of Interest 21 Source: Bowen et al. AJPM 2009
  22. 22. Summary – What is IR? / IR Outcomes • Implementation research is the scientific inquiry into questions concerning implementation • Such research could be conducted by observing changes in implementation research outcomes (IRO) during the implementation process • IRO describe effects (results) of a deliberate and purposive action to implement an intervention • IRO are mediators of individual/population health outcomes • IRO could comprise multiple domains or a single domain, are defined and measured in context across single or multiple socioecological levels • They could be measured using quantitative, qualitative and mixed method approach • Some common examples of IRO are fidelity, acceptability and feasibility 22
  23. 23. FORMULATING IMPLEMENTATION RESEARCH PROBLEMS AND QUESTIONS 23
  24. 24. 24 Addressing Public Health Problems Define the problem Measure the magnitude of the problem Develop a conceptual framework for the understanding of the key determinants of the problem Identify and develop strategies for intervention (program) Set priorities and recommend interventions (policy) Implement interventions and evaluate outcomes Communication strategy
  25. 25. 25 Problem Definition Creating a problem definition is an iterative process • Non-linear process • Changes in the definition are expected • Requires informed discussion • Tests team skills – Key difference with IR is the involvement of end users in defining the public health problem Defining a problem is an iterative process and is often the most difficult step in the problem solving methodology, but a good problem definition goes a long way towards successfully solving the problem http://www.notablebiographies.com/De-Du/Dewey-John.html
  26. 26. Prioritization of Health Issues for IR • Priority Rating Formula (Hanlon) – Component A = Size of the problem – Component B = Seriousness of the problem – Component C = Estimated effectiveness of the solution – Basic Priority Rating • BPR = (A + 2B) * C – +/- Component D = PEARL factors (propriety, economic, acceptability, resource availability, legality) – Overall Priority Rating • OPR = [(A + 2B) * C]D
  27. 27. 27 Specificity of the Problem • What is the specific dimension or aspect of interest? • If you think you are being too specific, you probably are not • Avoid broad, general topics as they run the risk of containing competing concepts – Eg. How do improve health systems in LMIC? – How to improve pediatric polio vaccine delivery among rural populations born in 2015 in Northern Nigeria?
  28. 28. Key Aspects of a Problem Definition • Nature of the problem/focus • Framing of the problem – Person – Place – Time • Specificity of the problem • Causality • Language considerations 28
  29. 29. 29 Framing of the Problem Person • Who is affected by this problem? • Be as specific as possible… – …using basic demographics: • Gender • Age • Race/ethnicity • Socioeconomic status (education, income, etc) – …and using problem-specific demographics • Eg. HIV – Sexual practices – Drug use – Housing situation – Occupation Estimates of new HIV Infections in the US, by Transmission Category, 2006 Hall HI, Song R, Rhodes P, et al. Estimation of HIV Incidence in the US. JAMA 2008;300: 520–529.
  30. 30. Place • Where is the problem located? • Be as specific as possible: – Geographical – Cultural – Social – Political – Economic – Historical – Ethnic 30 Framing of the Problem Cholera cases (dots) and outbreaks (orange), 2007-2009 http://gamapserver.who.int/mapLibrary/Files/Maps/Global_CholeraCases0709_20091008.png
  31. 31. Time • What is the specific time frame for the problem? • Be as specific as possible – Dates (1990-2000) – Time since a specific event (post 9/11) 31 Framing of the Problem http://www.cdc.gov/Features/dsSpinaBifida/
  32. 32. Summary Problem Statement • Summarizes the purpose of the study • Establishes the direction and captures the essence of the study • Reflects the research design of the study • Leads logically to the implementation research objectives/questions/specific aims 32
  33. 33. IR Questions • Of interest to stakeholders – Government, community, academia, implementing partners • Answerable • Shaped by the problem and in turn shape the design of the research • Clear, specific, and feasible • Three Broad Areas of IR Questions – Describe Health Situation or Interventions – Provide Data to Evaluate Ongoing Interventions or Information Needed to Adjust Interventions – Analyze missed targets and potential solutions 33
  34. 34. 34 Language Considerations • Be clear and concise • Define and reference technical terms • Define local terms • Generally a paragraph rather than a single statement • Incorporate general objectives and use action words to succinctly outline the purpose of the study – Purpose, Intent, Objective
  35. 35. 35 Causal Claims • Are etiologic factors or causes part of your problem statement? • AVOID causal claims • Be short and clear – this is the problem definition, not the answer to the problem
  36. 36. Small Group Assignment • Statement of the problem for your IR proposal – 1-2 paragraphs • Formulate one specific IR objective (question) – Your IR objective should include one of the IR outcomes discussed 36

×