A methodologically oriented systematic review was conducted to study current practices concerning the assessment of intervention fidelity in CRTs of public health interventions conducted in LMICs.

1. Comparison of registered and published
intervention fidelity assessment
in cluster randomised trials of
public health interventions
in low- and middle-income countries:
systematic review
Myriam Cielo Pérez, Nanor Minoyan, Valéry Ridde, Marie-Pierre Sylvestre and Mira Johri
Fifth Global Symposium on Health Systems Research, October 2018, Liverpool, UK.

2. Systematic
review:
Intervention
fidelity
assessment
in cluster
randomised
trials of public
health
interventions
1. Contexte
2. Objetive
v
3. Method
4. Results
5. Findings
6. Recommendations

3. • Cluster-randomised trials (CRTs)
Key tools to evaluate public health interventions
• CRTs of complex public health interventions conducted
in low- and middle-income countries (LMICs) are
prone to deficiencies in implementation
1. Contexte

4. • Intervention fidelity (IF) assessment
Component of process evaluation that aims to understand
and measure to what extent interventions are implemented
as planned
• CRT reporting guidelines: no recommendations for IF
1. Contexte

5. We conducted a methodologically oriented systematic
review to study current practices concerning the
assessment of intervention fidelity in CRTs of public health
interventions conducted in LMICs.
2. Objetive
v

6. Research questions:
• What proportion of recent CRTs of public health interventions
in LMICs planned to assess intervention fidelity (IF)?
• What proportion of recent CRTs of public health interventions
in LMICs reported assessing IF?
• Which fidelity components were examined, and which data
collection methods were employed to assess each
component?
• Is there evidence of divergent practices between planned and
reported studies?
2. Objetive
v

7. Participants +
Settings
Interventions Comparators Outcomes
Human adults +
children living in
LMICs
Public health
interventions
As defined by
CRT design
**protocol and trial
reporting of IF
assessment
**Studies that proposed methods to assess / reported key
fidelity components*:
Content
Coverage
Frequency
Duration
 Trial protocol and the main trial report (other linked
document)
3. Method
*Carroll et al (2007) & Hasson (2010)

8. 3. Method
• Databases: PubMed / MEDLINE, EMBASE, CINAHL,
EMB Reviews
• Keyword concept groups: CRTs, developing countries,
public health interventions
• Limits: - Jan 2012-May 2016
- Eng-Fr-Spa
- regist* in abstract
• Risk of bias Tool (Cochrane )

9. Flow diagram-study selection process4. Results

10. General characteristics of included studies (n=65)
4. Results

11. Fidelity assessment among CRTs of public health interventions in LMICs (n=90)
4. Results
CRTs of public health interventions in LMICs that planned / reported assessing IF
Trial protocols 40% (36/90)
Trial reports 71.1% (64/90)
Studies that did not assess any IF dimension in either the trial or protocol: 27.8% (25/90)

12. Fidelity assessment among CRTs of public health interventions in LMICs (n=90)
The proportion of overall agreement between the trial protocol and trial report concerning
occurrence of IF assessment was 66.7% (60/90)
4. Results

13. 4. Results
The proportion of positive agreement among those that agree was 58.3% (35/60)

14. Fidelity components
• Content
• Coverage
Data collection methods
• Field notes, self-reported evaluation or checklists
• Consultation of manuals
4. Results

15. • Fidelity assessment is not currently a systematic
practice in CRTs of public health interventions
carried out in LMICs.
• The quality of the components examined and data
collection methods were variable between studies.
5. Findings

16. • Development initiatives require high-quality
evaluations to determine whether the programs
work or not and to know how to improve them
• Improved assessment and reporting of intervention
fidelity is important to researchers, funders and
decision-makers
5. Findings

17. 1. CRT reporting guidelines should promote fidelity
assessment as a mean to facilitate valid causal
inference and replication of results.
2. Developing a practical tool to facilitate assessment
of implementation fidelity is necessary to guide
researchers in how to conduct IF assessment
6. Recommendations

18. To go further
1. Eldridge S, Ashby D, Bennett C, Wakelin M, Feder G. Internal and external
validity of cluster randomised trials: systematic review of recent trials. BMJ.
2008;336(7649):876–80.
2. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual
framework for implementation fidelity. Implement Sci. 2007;2(1):40
3. Hasson H. Systematic evaluation of implementation fidelity of complex
interventions in health and social care. Implement Sci. 2010;5(1):67.
4. Andrade S, Lachat C, Ochoa-aviles A, et al. A school-based intervention improves
physical fitness in Ecuadorian adolescents: a cluster-randomized controlled trial.
Int J Behav Nutr Phys Act. 2014;11:153.
5. Ayles H, Muyoyeta M, Du toit E, et al. Effect of household and community
interventions on the burden of tuberculosis in southern Africa: the ZAMSTAR
community-randomised trial. Lancet. 2013;382(9899):1183-94.
Pérez, M. C., Minoyan, N., Ridde, V., Sylvestre, M. P., & Johri, M. (2018). Comparison
of registered and published intervention fidelity assessment in cluster randomised
trials of public health interventions in low-and middle-income countries: systematic
review. Trials, 19(1), 410.

20. *Carroll et al (2007) & Hasson (2010)

21. Hasson 2010

22. Summary of risk of bias in included RCTs
Results

23. Fidelity components and moderating factors identified* (n=65)
Results