This document discusses realist evaluation and its application in implementation science. It begins by advocating for the use of theory in implementation sciences and the interest in understanding the "mechanisms" underlying change. Realist evaluation focuses on identifying contextual influences and articulating the mechanisms of implementation through program theories. Program theories in realist evaluation specify what mechanisms will generate outcomes in specific contexts. The document provides examples of program theories and outlines how they can be used as planning and evaluation tools. It also discusses sources for eliciting initial program theories and the process of program theory elicitation.

1. Realist evaluation: Its uses and
application in implementation science
Ferdinand C Mukumbang, PhD
Assistant Professor
Department of Global Health

2. Appendix
Reflections on last lesson

3. Common types of evaluations

4. The action model
The assumptions, theories and/or
knowledge that inform (and explain)
the design and implementation of
the intervention, and its outcomes
( // logic model, programme logic)
The causal model
Specifies the underlying causal
mechanisms in terms of
relationships between intervention
and outcome, the influence of
context and intervening factors
The elements of the program theory
Two dimensions

5. • Strong advocacy for the use of theory in Implementation
sciences, a spillover from the field of evaluation
• Interest in ‘mechanisms’ underlying change
• By identifying contextual influences and articulating
the mechanisms of implementation, theories can be
invaluable for explaining the roles of implementation
strategies and predicting how implementation may
unfold.
Mechanism-based theorizing

6. Appendix
Back to today’s lesson

7. Objectives
At the end of the class, participants should be able to
• Describe the core principles of realist evaluation and its
application in implementation science.
• Students will be able to articulate the usefulness and
drawbacks of realist evaluation in implementation science.

8. Focus on social science theories

9. Background on Realist Evaluation

10. Background on Realist Evaluation

11. IS requires studying mechanisms of
implementation.
Moderator
(cognitive)
Precondition
for
mechanism
activation
Precondition
for proximal
outcome
Change
mechanism
Moderator
(organization)
Proximal
outcome
Distal
outcome
Implementation
strategy
• Actor
• Action
• Temporality
• Dose
Lewis et al. (2018) ;
Weiner et. al. (2012)

12. Program theory
• Realist evaluation starts and ends with a theory or with theories.

13.  A set of explicit or implicit assumptions of how the program should
be organized and why the program is expected to work
 They describe mechanisms introduced by the program into pre-
existing contexts that can generate outcomes
Program theory

14. What is a program theory
Program theory in realist evaluation specifies what
mechanisms will generate the outcomes in what context.

15. What is a program theory
Program theory in realist evaluation specifies what
mechanisms will generate the outcomes in what context.
Duprez, V., Dhont, L., van der Cingel, M., Hafsteinsdóttir, T. B., & Malfait, S. (2023). Understanding strategies that foster nurses to act as clinical leaders in hospitals: A realist
review. Journal of Advanced Nursing, 00, 1–14. https://doi.org/10.1111/jan.15902

16. Uses of IPTs
As a planning tool
The program theory
indicates how different
elements of the program are
intended to work together
and to identify the
intermediate outcomes of a
program or an intervention

17. Eliciting an IPT

18. Eliciting an IPT
IF a population segmentation tool is provided to PCPs,
THEN PCPs can successfully identify chronically ill patients who most likely
have complex biopsychosocial needs for a PCNA,
BECAUSE PCPs gained increased confidence in identifying chronically ill
patients who most likely have complex biopsychosocial needs, in the context
of a heterogeneous chronically ill patient population.

19. Eliciting an IPT
IF PCPs are offered a PCNA conversation tool and training,
THEN PCPs are likely to enhance their person-centered conversational skills,
BECAUSE the tool and training instill confidence in PCPs to enhance these skills.
THEN, in a context where PCPs are adequately skilled in guiding a PCNA, PCPs are
enabled to engage in shared decision making about the required tailored care, referral,
and coordination

20. Eliciting an IPT
IF PCPs are offered a PCNA
conversation tool and training,
THEN PCPs are likely to enhance
their person-centered conversational
skills,
BECAUSE the tool and training instill
confidence in PCPs to enhance these
skills.
THEN, in a context where PCPs are
adequately skilled in guiding a PCNA,
PCPs are enabled to engage in
shared decision making about the
required tailored care, referral, and
coordination

21. As an evaluation tool
 The goal of the evaluator(s) is to understand not only patterns
related to the outcome of the intervention but also to reveal how
and why the intervention attains the outcome of interest
Uses of IPTs continued

22. Sources of information for IPT
Ferdinand C Mukumbang et al. BMJ Open 2016;6:e009977

23. Assignment Discussion
• Assignment – Session 10
• Using the implementation science strategy described in Assignment 9, design a
realist evaluation protocol in less than 500 words.
• Outline the phases you will follow for your evaluation (Max 50 words).
• Describe the data collection and analysis approaches that you will use at the
various stages (phases) of your evaluation (Max 400 words).
• Describe the possible significance of your findings to program implementers and
policymakers (max 50 words)

24. Process of program theory elicitation
Program theory in realist evaluation specifies what mechanisms
will generate the outcomes in what context.

25. IPT 1

26. Fig 2. CMO for providers.
Mukumbang FC, van Belle S, Marchal B, van Wyk B (2016) Towards Developing an Initial Programme Theory: Programme
Designers and Managers Assumptions on the Antiretroviral Treatment Adherence Club Programme in Primary Health Care Facilities
in the Metropolitan Area of Western Cape Province, South Africa. PLOS ONE 11(8): e0161790. doi:10.1371/journal.pone.0161790
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0161790

27. IF the intervention groups patients with particular interpersonal characteristics
(health-literate adults (18+) and provides quick symptom checks, quick access to
medication, consistent counselling and social support from the peer counsellor,
THEN patients are likely to adhere to medication and remain in care,
BECAUSE they develop a group identity which improves their perceived social
support, satisfaction and trust; and acquire knowledge which helps them to
understand their perceived threat and perceived benefits and improves their self-
efficacy.
As a result, they become encouraged, empowered and motivated, thus, more likely
to remain in care and adhere to the treatment.
THEORY 1

28. THEORY 2
IF operational staff receive goals and targets set to continuously enrol patients
in the adherence club and strictly monitor their participation through strict
standard operating practices (pill counting, and the promise of exclusion in the
event of missed appointment and active patient tracing),
THEN patients are likely to adhere to medication and remain in care,
BECAUSE they fear of losing the benefits (easy access to medication, peer
support, reduced waiting times, and two-month ART collection) of the club
system and they are coerced through adhesive club rules.
As a result, they become nudged to remain in care and adhere to the
treatment which might decongest the health facility.

29. Process of IPT elicitation Ex 2
Program theory in realist evaluation specifies what mechanisms
will generate the outcomes in what context.

30. Initial program theory of the Health Homes and Neighbourhood

31. Thank you!