This document provides an overview of implementation research (IR), defining what it is and how it relates to other forms of research. It discusses key IR outcomes like acceptability, feasibility, and fidelity that are commonly studied. The document aims to help participants explain IR, design answerable IR questions, and understand how to measure various IR outcomes. It covers topics like formulating IR problems, prioritizing health issues for research, and developing specific IR objectives and questions. The presentation provides frameworks and examples to help participants better understand IR concepts and apply them to their own work.

1. Implementation Research:
What is it? Am I doing it?
How can I do it better?
Olakunle Alonge, Assistant Scientist,
Health Systems Program, JHBSPH
Jim Ricca, MCSP Learning and
Implementation Science Team
Leader
Stefan Baral, JHBSPH

2. Learning Objectives
By the end of this session, participants will be able to:
• Explain what Implementation Research (IR) is and how it
related to other forms of research
• Explain three commonly studied Implementation Research
outcomes
• Design answerable Implementation Research questions of
importance to key stakeholders
2

3. Implementation Practice and Research
• “Implementation [practice] is the act of carrying an
intention into effect, which in health can be policies,
programmes, or individual practices (collectively
called interventions)” e.g. the act of administering
misoprostol for AMTSL
• “Implementation research is the scientific inquiry
into questions concerning implementation” e.g. study
on the acceptability of misoprostol among pregnant
women in an area
3 Source: Peters, Adam, Alonge, Agyepong and Nhan, BMJ 2013; Gilbert, JRScT 1991

4. Implementation Research
Translational Research
Implementation Research and Other
Common Studies in Public Health
4
T4
studies
T3
(D&I)
studies
Intervention studies
T2
studies
T1
studies
QI studies
Operation
Research
M&E
Source: Lobb and Colditz, Annual Rev Pub Health 2013; IRDS Consultative Meeting Report, 2014

5. IMPLEMENTATION OUTCOMES:
FRAMEWORK AND DEFINITIONS
5

6. Implementation Outcomes
• Implementation variables:
– Constructs whose vector properties (magnitude and direction)
changes with an implementation process
• Implementation outcomes:
– Implementation variables that describe effects (results) of a deliberate
and purposive action to implement an intervention
– Useful for measuring success or failure of an implementation process
– Intermediate outcomes of the effectiveness of an intervention (on
individual/population health outcomes and quality of life)
– Underdeveloped constructs; operates across socio-ecological levels
6 Source: Peters, Adams, Alonge et al 2013; Proctol et al. 2011; Fixsen et al. 2005

7. Conceptual Framework for Implementation
Outcomes
Clients Outcome
Satisfaction
Symptomatology
Function
Population-
Based
Incidence of
diseases
Morbidity
Mortality
DALYs
Health
Outcomes
Efficiency
Coverage
Equity
Responsiveness
Services
Outcomes**
Acceptability
Adoption
Appropriateness
Costs
Feasibility
Fidelity
Penetration
Sustainability
Implementation
Outcomes I*
7
• *measurable both at the individual and aggregated level
• **measurable mostly at an aggregated level
Source: Proctor et al 2011

8. Implementation Outcomes - Definitions
Implementation
Outcome
Working Definition* Related terms**
Acceptability The perception among stakeholders
(e.g. consumers, providers, managers,
policy-makers) that an intervention is
agreeable
Related factors: (e.g.
Comfort, Relative
advantage, Credibility)
Adoption The intention, initial decision, or action
to try to employ a new intervention
Uptake, Utilization,
Intention to try,
Appropriateness The perceived fit or relevance of the
intervention in a particular setting or for
a particular target audience (e.g.
provider or consumer) or issue
Relevance, Perceived fit,
Compatibility, Perceived
usefulness or suitability
Feasibility The extent to which an intervention can
be carried out in a particular setting or
organization
Practicality, Actual fit,
Utility, Trialability
8 Source: Proctor et al 2011; Peters, Adams, Alonge et al 2013

9. Implementation Outcomes - Definitions
Implementation
Outcome
Working Definition* Related terms**
Fidelity The degree to which an intervention was
implemented as it was designed in an original
protocol, plan, or policy
Adherence, Delivery as
intended, Integrity,
Quality of delivery,
Intensity or dosage of
delivery
Implementation
cost
The incremental cost of the implementation
strategy (i.e. how the services are delivered in a
particular setting); including the cost of the
intervention itself.
Marginal cost, Total
cost***
Penetration/
Coverage
Extent to which a practice or program is
integrated within a service settings and system
(niche saturation). The degree to which the
population that is eligible to benefit from an
intervention actually receives it.
Reach, Access, Service
Spread or Effective
Coverage, Penetration
Sustainability The extent to which an intervention is
maintained or institutionalized in a given setting.
Maintenance,
Continuation,
Routinization,
Institutionalization,
9 Source: Proctor et al 2011; Peters, Adams, Alonge et al 2013

10. Measuring Implementation Outcomes
10 Source: Proctor et al 2011

11. Measuring Implementation Outcomes
11 Source: Proctor et al 2011

12. FIDELITY
12

13. Fidelity - Definition
• Degree to which an intervention was implemented as
prescribed by the original protocol or program developers
• Adherence to the core program components
– Core components: elements without which the intervention will not
have its intended effect
– Balancing fidelity and adaptation: 100% fidelity may be unrealistic
13 Source: Dusenbury et al 2003, Rabin et al. 2008

14. Fidelity – Domains of Interest
• May consist of the following domains:
14 Source: Dane & Schneider, 1998; Dusenbury et al 2003; Carroll et al. 2007; Durlak and Dupre 2008; Proctor et al 2011
Dane & Schneider,
1998
Carroll et al. 2007 Proctor et al. 2011 Durlak and DuPre
2008
Adherence Adherence Adherence Adherence
Dose Exposure or dose Dose
Quality of delivery Quality of delivery Quality of delivery
Participant
responsiveness
Participant
responsiveness
Program
differentiation
Program
differentiation
Intervention
complexity
Facilitation
strategies

15. Fidelity – Domains of Interest
• Domains of interest:
– Adherence – an intervention is being delivered as it was designed or written
– Dosage or exposure - frequency and duration of the intervention is as full as
prescribed by its designers. Could mean amount of intervention that the recipient
received
– Quality of delivery - the manner in which an intervention worker (e.g. teacher,
volunteer, or staff member) delivers a program
• Based on techniques prescribed by developer
• Based on a theoretical ideal
– Participant responsiveness - how far participants respond to, or are engaged by,
an intervention
– Program differentiation - identifying unique features of different components of
an intervention, and identifying the core components
• Perspectives in measurement:
– Representative (commonest approach)
– Comprehensive
– Complex
15 Source: Mihalic 2004, Carroll 2007

16. ACCEPTABILITY
16

17. Acceptability
• Perception among stakeholders (e.g. consumers, providers,
managers, policy-makers) that an intervention is agreeable
– Based on direct knowledge of or experience with specific aspects of
the intervention, e.g. content, comprehensiveness, ease of use,
comfort, etc.
– Distinguished from service satisfaction which often focuses on general
service experience
– Dynamic and changes with experience
• Often conceptualized as consisting a single domain
17

18. FEASIBILITY
18

19. Feasibility
• The extent to which an intervention can be carried out in a
particular setting or organization
– Typically assessed retrospectively to explain success or failure of an
implementation process (actual fit)
– Closely related to appropriateness which looks at the perceived fit of
an intervention for a setting (i.e. prospectively assessed)
• Has multiple domains and similar measurement perspectives
as in fidelity (i.e. representative, comprehensive, complex)
19

20. Feasibility – Domains of Interest
20 Source: Bowen et al. AJPM 2009

21. Feasibility – Domains of Interest
21 Source: Bowen et al. AJPM 2009

22. Summary – What is IR? / IR Outcomes
• Implementation research is the scientific inquiry into questions
concerning implementation
• Such research could be conducted by observing changes in
implementation research outcomes (IRO) during the implementation
process
• IRO describe effects (results) of a deliberate and purposive action
to implement an intervention
• IRO are mediators of individual/population health outcomes
• IRO could comprise multiple domains or a single domain, are
defined and measured in context across single or multiple
socioecological levels
• They could be measured using quantitative, qualitative and mixed
method approach
• Some common examples of IRO are fidelity, acceptability and
feasibility
22

23. FORMULATING IMPLEMENTATION RESEARCH
PROBLEMS AND QUESTIONS
23

24. 24
Addressing Public Health Problems
Define the problem Measure the magnitude of the problem
Develop a conceptual framework
for the understanding of the key
determinants of the problem
Identify and develop strategies for
intervention (program)
Set priorities and recommend
interventions (policy)
Implement interventions
and evaluate outcomes
Communication strategy

25. 25
Problem Definition
Creating a problem definition is an iterative
process
• Non-linear process
• Changes in the definition are expected
• Requires informed discussion
• Tests team skills
– Key difference with IR is the involvement of
end users in defining the public health
problem
Defining a problem is an iterative process and is often the
most difficult step in the problem solving methodology,
but a good problem definition goes a long way towards successfully solving the problem
http://www.notablebiographies.com/De-Du/Dewey-John.html

26. Prioritization of Health Issues for IR
• Priority Rating Formula (Hanlon)
– Component A = Size of the problem
– Component B = Seriousness of the problem
– Component C = Estimated effectiveness of the solution
– Basic Priority Rating
• BPR = (A + 2B) * C
– +/- Component D = PEARL factors (propriety, economic, acceptability,
resource availability, legality)
– Overall Priority Rating
• OPR = [(A + 2B) * C]D

27. 27
Specificity of the Problem
• What is the specific dimension or aspect of interest?
• If you think you are being too specific, you probably are not
• Avoid broad, general topics as they run the risk of
containing competing concepts
– Eg. How do improve health systems in LMIC?
– How to improve pediatric polio vaccine delivery among
rural populations born in 2015 in Northern Nigeria?

28. Key Aspects of a Problem Definition
• Nature of the problem/focus
• Framing of the problem
– Person
– Place
– Time
• Specificity of the problem
• Causality
• Language considerations
28

29. 29
Framing of the Problem
Person
• Who is affected by this problem?
• Be as specific as possible…
– …using basic demographics:
• Gender
• Age
• Race/ethnicity
• Socioeconomic status (education, income, etc)
– …and using problem-specific demographics
• Eg. HIV
– Sexual practices
– Drug use
– Housing situation
– Occupation
Estimates of new HIV Infections in the US, by Transmission
Category, 2006
Hall HI, Song R, Rhodes P, et al. Estimation of HIV Incidence in the US. JAMA 2008;300: 520–529.

30. Place
• Where is the problem located?
• Be as specific as possible:
– Geographical
– Cultural
– Social
– Political
– Economic
– Historical
– Ethnic
30
Framing of the Problem
Cholera cases (dots) and outbreaks (orange), 2007-2009
http://gamapserver.who.int/mapLibrary/Files/Maps/Global_CholeraCases0709_20091008.png

31. Time
• What is the specific time frame for the problem?
• Be as specific as possible
– Dates (1990-2000)
– Time since a specific event (post 9/11)
31
Framing of the Problem
http://www.cdc.gov/Features/dsSpinaBifida/

32. Summary Problem Statement
• Summarizes the purpose of the study
• Establishes the direction and captures the essence
of the study
• Reflects the research design of the study
• Leads logically to the implementation research
objectives/questions/specific aims
32

33. IR Questions
• Of interest to stakeholders
– Government, community, academia, implementing partners
• Answerable
• Shaped by the problem and in turn shape the design of the
research
• Clear, specific, and feasible
• Three Broad Areas of IR Questions
– Describe Health Situation or Interventions
– Provide Data to Evaluate Ongoing Interventions or Information
Needed to Adjust Interventions
– Analyze missed targets and potential solutions
33

34. 34
Language Considerations
• Be clear and concise
• Define and reference technical terms
• Define local terms
• Generally a paragraph rather than a single statement
• Incorporate general objectives and use action words to
succinctly outline the purpose of the study
– Purpose, Intent, Objective

35. 35
Causal Claims
• Are etiologic factors or causes part of your problem
statement?
• AVOID causal claims
• Be short and clear – this is the problem definition, not the
answer to the problem

36. Small Group Assignment
• Statement of the problem for your IR proposal
– 1-2 paragraphs
• Formulate one specific IR objective (question)
– Your IR objective should include one of the IR outcomes discussed
36