SlideShare a Scribd company logo
THEORY AND PRACTICE OF REALIST EVALUATION
Some thoughts about the theoretical foundations
Valéry Ridde, Emilie Robert, Anthony Lacouture
July, 2; 2015
WHO Meeting : Realist evaluation of national strategic planning and policy processes
equitesante.org
Equitesante.org
2
3
RESEARCH QUESTION
• WHAT ?
Equitesante.org
4
• WE ALREADY KNOW
WHAT WORKS…
FROM WHAT TO….
HOW
Equitesante.org
5
FOR WHOM
WHY
TO WHAT EXTENT
IN WHAT CIRCUMSTANCES
PUBLIC HEALTH AND COMPLEXITY
« Multiple entangled actions, acting in often
non-linear relationships, organized in a specific
context and understood as open systems,
researcher who observe in fact itself part of it »
Blaise et al. 2010
Equitesante.org
6
7 CHARACTERISTICS OF COMPLEX INTERVENTIONS
Volitions
Equitesante.org
7
Implementation
Contextes
Time
Outcomes
Rivalry
Emergence
Pawson, 2013
CRITICAL REALISM
Equitesante.org
8
POSITIVISM CRITICAL REALISM RELATIVISM
QUESTIONS
Is the policy or
intervention
(cost)-effective?
What works for
whom under
which conditions?
How do actors
experiences, power
and understand?
DISCIPLINES
Epidemiology
Welfare
economics
Political science
Policy analysis
Organizational
studies
Socio-anthropology
DESIGN
Deductive
Fixes
Abductive
Pragmatic/MM
Inductive
Flexibles
Gilson, 2012; Robson, 2002
CRITICAL REALISM
• Reality exists outside the human
constructions, but it is perceptible only
through our senses
Equitesante.org
9
Maxwell, Pawson, Bhaskar, etc.
GENERATIVE CAUSALITY
• We are not looking for immutable law vs
sucessionist notion of causality
• Importance of mechanisms/context vs
intervention
• But regularities (semi-reg : Lawson)
– semi-predictable patterns
– pathway of program functioning
• Patterns = CMO
Equitesante.org
10
(Pawson, 2006)
C M O
Equitesante.org
11
Robert & Ridde, 2014
MECHANISM DEFINITION
• is hidden but real, sensitive to the context and
produces outcomes
• is an element of reasoning and reactions of
agents in regard of the resources available in a
given context to bring about changes
• evolves within an open space-time and social
system of relationships
Equitesante.org
12
Lacouture et al, 2015
MECHANISM
• Elements of reasoning of
agents and their choices
about how the change
will be achieved through
the implemented
intervention
• Reactions of agents in
response to the resources
provided by the
intervention
13
Equitesante.org
• Pre-existing characteristics
of the individuals, localities,
situations or systems in
which an intervention is
being set up (= CONTEXT)
• Intentional measures,
activities or a rational plan
taken by program
implementers (= STRATEGIES
OF INTERVENTION)
Lacouture et al, 2015
Equitesante.org
14
Mobilisation of resources
NOT A MECHANISM
Equitesante.org
15
Actor’s attitudes : health workers’ motivation
and satisfaction (A1), the provider–patient
relationship (A2), patients’ satisfaction (A3),
patients’ perceptions on health and on financial
access to healthcare (A4), and health workers’
coping strategies (A5).
Ridde et al, 2012
MECHANISM EXAMPLE
CMOc of PR synergy related to interpreting and
disseminating research findings (strategy).
•As a result of the long-term collaborative relationships
between academic and community stakeholders (context),
locally informed and accessible scientific study findings were
presented to the community (strategy). The audiences at
the presentations felt comfortable and safe participating in
the context of their community ownership of the project
(mechanism) and made sense of the data in their own
terms. As a result, new explanatory insights into the data
were generated (outcome).
Equitesante.org
16
Jagosh et al, 2014
THEORY-DRIVEN EVALUATION
• Program/intervention theory (Weiss, Chen, etc.. )
• Causal model
• Middle range theory (initial and final)
Equitesante.org
17
INTERVENTION THEORY
• "A program is a theory and an evaluation is its
test" (Rein, 1981)
• Programs are “theories incarnate”. (Pawson 2012)
• “It is a small theory attempting explanation of
specific treatment processes, not a large theory
of general social phenomena” (Lipsey, 1993)
Equitesante.org
18
19
INITIAL PT IN UGANDA
• Management practices and leadership styles (interven-
tion) that are supportive of autonomy lead to positive work
behaviour of volunteers, such as improved task per-
formance and persistence (outcomes), because they satisfy
the psychological needs of autonomy, competence and
relatedness (mechanism). More specifically, such
management and leadership contribute to a shift from
‘external motivation’ towards ‘internalised motivation’ for
the volunteers whose initial engagement was more driven
by external motives. Internalised motivation leads not only
to positive work behaviour, but also to well- being. Such
management and leadership furthermore allow volunteers
whose initial engagement was mainly driven by internal
motives to remain engaged and perform well.
Equitesante.org
20
Vareilles et al, 2015
MIDLE RANGE THEORY
• « Middle-range theory involves abstraction, of
course, but they are close enough to observed
data to be incorporated in propositions that
permit empirical testing. » Merton, 1967
• Level of abstraction = useful + testable/data
Equitesante.org
21
INITIALE MIDLE RANGE THEORY
• "Hospital managers of well-performing hospitals
deploy organisational structures that allow
decentralisation and self-managed teams and
stimulate delegation of decision-making, good
flows of information and transparency. Their
HRM bundles combine employment security,
adequate compensation and training. This results
in strong organisational commitment and trust.
Conditions include competent leaders with an
explicit vision, relatively large decision-making
spaces and adequate resources."
Equitesante.org
22
Marchal et al, 2010
INITIALE MIDLE RANGE THEORY
• "‘A pluralistic health system harbours a web of accountability relationships
between actors who combine the roles of account-holder and accountor. 
• Public accountability is actualised when actors are answerable to the public
and remedial action is undertaken. Public accountability requires both
answerability and enforceability in order to be actualised. The answerability or
the capability of the DHMT, SRH INGOs and partnerships to inform, evaluate
and report in an open manner requires transparency and clarity on whom
they represent and deliver services to. Answerability is actualised through
practices grounded in compliance and persuasion.
• Enforceability is grounded in the capability of the public to
demand accountability on the one hand and in meta-governance, i.e. the
function, exercised by a state actor(s), of regulating, monitoring and
sanctioning on the public’s behalf, on the other hand.
• Accountability practices operate along four dimensions (social, political,
organisational and the provider dimension). Each dimension has specific
bundles of strategies, practices, relationships and outcomes. Accountability is
embedded in vertical, horizontal and partnership governance arrangements.
• Multi-level governance arrangements weaken public accountability when
there is confusion over roles and responsibilities between governing actors.’"
Equitesante.org
23
Van Belle, 2014
CMO : HEURISTIC FRAMEWORK
Ugly
circumlocution
(Pawson et al, 2012)
Equitesante.org
24Robert & Ridde, 2014
Original CMO for the CHW
• Context
– Lack of community awareness regarding safe motherhood,
danger signs during pregnancy, labour, and postpartum
periods and where to go for services.
• Programme response/mechanism
– Training and deploying trusted members of the
community instills ‘knowledge’ of dangersigns during
pregnancy and labour, safe delivery practices, as well as
newborn care. Regular visits from community members
build relationships between women and health workers,
leading to ‘trust’ in the health system.
• Outcome
– More women aware of complications; increased care
seeking.
Equitesante.org
25Adams et al, 2015
Initiale CMO CONFIGURATION
Equitesante.org
26
Evans et al, 2014
Final CMO CONFIGURATION
Equitesante.org
27Adams et al, 2015
CMO CONFIGURATION
A community experiences a high level of
unemployment to which an employment
training program is offered in a remote part of
town (context). But the program has low
enrollment and high attrition, and few people
are trained (outcome). The reason is that people
feel disillusioned by the lack of effort by
program planners to ensure adequate public
transportation to the venue (mechanism).
Equitesante.org
28
Jagosh et al, 2013
Initiale CMO CONFIGURATION
Participants who are motivated and share a
good rapport’ (context) may ‘share ideas,
experiences and support one another’
(mechanism), which may be associated with a
smoking cessation and improved weight
management (outcome)
Equitesante.org
29
Mackenzieet al, 2009
THEORIES AND CMO
volunteer motivation & work behaviour
• Crowding theory
• Intrinsic motivation theory
• self-determination theory
• Gift exchange and high-
performance HRM theory
• Consistency theory
• Organisational
commitment
• Social exchange theorY
participatory research assessment
30
Equitesante.org
Vareilles et al, 2015
Jagosh et al, 2013
• Theory of Capacity Building
• Communicative Action
Theory
• Empowerment/emancipati
on
• Collaborative Management
• Partnership Synergy Theory
FROM CMOi TO CMOf
Equitesante.org
31
Pawson et al, 2012
32
33
34
CONTEXTUAL LAYERS
Equitesante.org
35
Robert & Ridde, 2014
MECHANISMS CHALLENGES
Equitesante.org
36
• a sense of self efficacy,
• an increase in self esteem,
• assurance that there is a system for back-up
support
MECHANISMS CHALLENGES
Equitesante.org
37
38
Robert & Ridde, 2014
REALIST EVALUATION CYCLE
Equitesante.org
39Vareilles et al, 2015
FROM CMO TO MRT
Equitesante.org
40
Abstraction
Empirical
MRT1
THEORY1
C1
M1
O1
PROGRAM THEORY
C2
M2
O2
C3
M3
O3
C4
M4
O4
C1
M1
O1
C1
M2
O3
C1
M1
O3
C2
M2
O1
C1
M3
O3
C1
M4
O3
MRT2
MRT3
MRT4
MRT5
THEORY2
THEORY3
STARTSTART
FROM CMO TO MRT
Equitesante.org
41
Abstraction
Empirical
MRT1
THEORY1
C1
M1
O1
PROGRAM THEORY
C2
M2
O2
C3
M3
O3
C4
M4
O4
C1
M1
O1
C1
M2
O3
C1
M1
O3
C2
M2
O1
C1
M3
O3
C1
M4
O3
MRT2
MRT3
MRT4
MRT5
THEORY2
THEORY3
STARTSTART
MERCI
equitesante.org
@equitesante

More Related Content

What's hot

Introduction to monitoring and evaluation
Introduction to monitoring and evaluationIntroduction to monitoring and evaluation
Introduction to monitoring and evaluation
Meshack Lomoywara
 
Monitoring evaluation
Monitoring evaluationMonitoring evaluation
Monitoring evaluation
Carlo Magno
 
Narrative inquiry
Narrative inquiryNarrative inquiry
Narrative inquiry
Pun Yanut
 
historica research
historica researchhistorica research
historica research
Nighat Shaheen
 
Monitoring and Evaluation Policy
Monitoring and Evaluation PolicyMonitoring and Evaluation Policy
Monitoring and Evaluation Policy
Komal Zahra
 
Results Based Monitoring and Evaluation
Results Based Monitoring and EvaluationResults Based Monitoring and Evaluation
Results Based Monitoring and Evaluation
Madhawa Waidyaratna
 
Theory of Change
Theory of ChangeTheory of Change
Theory of Change
clearsateam
 
Good Governance in the Public Sector Presentation
Good Governance in the Public Sector PresentationGood Governance in the Public Sector Presentation
Good Governance in the Public Sector Presentation
International Federation of Accountants
 
Qualitative Research Methodology - Ethnography
Qualitative Research Methodology - EthnographyQualitative Research Methodology - Ethnography
Qualitative Research Methodology - Ethnography
Bryll Edison Par
 
Dissertation presentation
Dissertation presentation Dissertation presentation
Dissertation presentation
Gregory McCaig
 
Multi-Stakeholder Partnership (MSP)
Multi-Stakeholder Partnership (MSP)Multi-Stakeholder Partnership (MSP)
Multi-Stakeholder Partnership (MSP)
Shipra Sharma
 
Better theories
Better theoriesBetter theories
Better theories
yee tandog
 
Monitoring and evaluation presentatios
Monitoring and evaluation presentatios Monitoring and evaluation presentatios
Monitoring and evaluation presentatios
athanzeer
 
Où en est la recherche sur le TC?
Où en est la recherche sur le TC?Où en est la recherche sur le TC?
Où en est la recherche sur le TC?
Equipe RENARD
 
Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统
Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统
Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统
Dadang Solihin
 
Research paradigm: Critical Realism in Burrell and Morgan quadrants
Research paradigm: Critical Realism in Burrell and Morgan quadrantsResearch paradigm: Critical Realism in Burrell and Morgan quadrants
Research paradigm: Critical Realism in Burrell and Morgan quadrants
Laban Bagui
 
MBA thesis presentation Nepal open university
MBA thesis presentation Nepal open universityMBA thesis presentation Nepal open university
MBA thesis presentation Nepal open university
Rabindra Aryal
 
Action Research
Action ResearchAction Research
Action Research
AnmOl Ali
 
Result based management
Result based management Result based management
Result based management
Tilak Raj Chaulagai
 
Quantitative paradigm in research
Quantitative paradigm in researchQuantitative paradigm in research

What's hot (20)

Introduction to monitoring and evaluation
Introduction to monitoring and evaluationIntroduction to monitoring and evaluation
Introduction to monitoring and evaluation
 
Monitoring evaluation
Monitoring evaluationMonitoring evaluation
Monitoring evaluation
 
Narrative inquiry
Narrative inquiryNarrative inquiry
Narrative inquiry
 
historica research
historica researchhistorica research
historica research
 
Monitoring and Evaluation Policy
Monitoring and Evaluation PolicyMonitoring and Evaluation Policy
Monitoring and Evaluation Policy
 
Results Based Monitoring and Evaluation
Results Based Monitoring and EvaluationResults Based Monitoring and Evaluation
Results Based Monitoring and Evaluation
 
Theory of Change
Theory of ChangeTheory of Change
Theory of Change
 
Good Governance in the Public Sector Presentation
Good Governance in the Public Sector PresentationGood Governance in the Public Sector Presentation
Good Governance in the Public Sector Presentation
 
Qualitative Research Methodology - Ethnography
Qualitative Research Methodology - EthnographyQualitative Research Methodology - Ethnography
Qualitative Research Methodology - Ethnography
 
Dissertation presentation
Dissertation presentation Dissertation presentation
Dissertation presentation
 
Multi-Stakeholder Partnership (MSP)
Multi-Stakeholder Partnership (MSP)Multi-Stakeholder Partnership (MSP)
Multi-Stakeholder Partnership (MSP)
 
Better theories
Better theoriesBetter theories
Better theories
 
Monitoring and evaluation presentatios
Monitoring and evaluation presentatios Monitoring and evaluation presentatios
Monitoring and evaluation presentatios
 
Où en est la recherche sur le TC?
Où en est la recherche sur le TC?Où en est la recherche sur le TC?
Où en est la recherche sur le TC?
 
Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统
Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统
Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统
 
Research paradigm: Critical Realism in Burrell and Morgan quadrants
Research paradigm: Critical Realism in Burrell and Morgan quadrantsResearch paradigm: Critical Realism in Burrell and Morgan quadrants
Research paradigm: Critical Realism in Burrell and Morgan quadrants
 
MBA thesis presentation Nepal open university
MBA thesis presentation Nepal open universityMBA thesis presentation Nepal open university
MBA thesis presentation Nepal open university
 
Action Research
Action ResearchAction Research
Action Research
 
Result based management
Result based management Result based management
Result based management
 
Quantitative paradigm in research
Quantitative paradigm in researchQuantitative paradigm in research
Quantitative paradigm in research
 

Viewers also liked

Mobile Ready Online - Move Forward with Mobile Websites
Mobile Ready Online - Move Forward with Mobile WebsitesMobile Ready Online - Move Forward with Mobile Websites
Mobile Ready Online - Move Forward with Mobile Websites
Natalie Norris Online Marketing
 
The Black Lab at the University of Kentucky
The Black Lab at the University of KentuckyThe Black Lab at the University of Kentucky
The Black Lab at the University of Kentucky
Sherif El Refai
 
Wmp 10 dec2012
Wmp 10 dec2012Wmp 10 dec2012
Wmp 10 dec2012
fincorCorretora
 
Black money
Black moneyBlack money
Black money
mehaksukhramani28
 
Donations for the Livestrong Foundation
Donations for the Livestrong FoundationDonations for the Livestrong Foundation
Donations for the Livestrong Foundation
Sherif El Refai
 
4 Ingredients to Successful Landing Pages
4 Ingredients to Successful Landing Pages4 Ingredients to Successful Landing Pages
4 Ingredients to Successful Landing Pages
Brian Lewis
 
Dressesmall cocktail dresses australia
Dressesmall cocktail dresses australiaDressesmall cocktail dresses australia
Dressesmall cocktail dresses australia
jane jones
 
The « Concept mapping » methodology: a review of users’ evaluative comments. ...
The « Concept mapping » methodology: a review of users’ evaluative comments. ...The « Concept mapping » methodology: a review of users’ evaluative comments. ...
The « Concept mapping » methodology: a review of users’ evaluative comments. ...
valéry ridde
 
Recent Victories at Haworth, Coleman & Gerstman, LLC
Recent Victories at Haworth, Coleman & Gerstman, LLC Recent Victories at Haworth, Coleman & Gerstman, LLC
Recent Victories at Haworth, Coleman & Gerstman, LLC
Scott Haworth
 
Software savings world reality
Software savings world realitySoftware savings world reality
Software savings world reality
softwaresavingsworld
 
Software savings world office 2010
Software savings world office 2010Software savings world office 2010
Software savings world office 2010
softwaresavingsworld
 

Viewers also liked (14)

Mobile Ready Online - Move Forward with Mobile Websites
Mobile Ready Online - Move Forward with Mobile WebsitesMobile Ready Online - Move Forward with Mobile Websites
Mobile Ready Online - Move Forward with Mobile Websites
 
The Black Lab at the University of Kentucky
The Black Lab at the University of KentuckyThe Black Lab at the University of Kentucky
The Black Lab at the University of Kentucky
 
Wmp 10 dec2012
Wmp 10 dec2012Wmp 10 dec2012
Wmp 10 dec2012
 
Black money
Black moneyBlack money
Black money
 
Donations for the Livestrong Foundation
Donations for the Livestrong FoundationDonations for the Livestrong Foundation
Donations for the Livestrong Foundation
 
4 Ingredients to Successful Landing Pages
4 Ingredients to Successful Landing Pages4 Ingredients to Successful Landing Pages
4 Ingredients to Successful Landing Pages
 
Dressesmall cocktail dresses australia
Dressesmall cocktail dresses australiaDressesmall cocktail dresses australia
Dressesmall cocktail dresses australia
 
The « Concept mapping » methodology: a review of users’ evaluative comments. ...
The « Concept mapping » methodology: a review of users’ evaluative comments. ...The « Concept mapping » methodology: a review of users’ evaluative comments. ...
The « Concept mapping » methodology: a review of users’ evaluative comments. ...
 
Kora
KoraKora
Kora
 
Recent Victories at Haworth, Coleman & Gerstman, LLC
Recent Victories at Haworth, Coleman & Gerstman, LLC Recent Victories at Haworth, Coleman & Gerstman, LLC
Recent Victories at Haworth, Coleman & Gerstman, LLC
 
Software savings world reality
Software savings world realitySoftware savings world reality
Software savings world reality
 
White
WhiteWhite
White
 
Mama pointer
Mama pointerMama pointer
Mama pointer
 
Software savings world office 2010
Software savings world office 2010Software savings world office 2010
Software savings world office 2010
 

Similar to Realist Evaluation : some thoughts about the theoretical foundations

HFG Health Governance Presentation at 2015 USAID Global Health Mini-University
HFG Health Governance Presentation at 2015 USAID Global Health Mini-UniversityHFG Health Governance Presentation at 2015 USAID Global Health Mini-University
HFG Health Governance Presentation at 2015 USAID Global Health Mini-University
HFG Project
 
KEYSTONE / Module 6 / Slideshow 3 / Researching Health Policy
KEYSTONE / Module 6 / Slideshow 3 / Researching Health PolicyKEYSTONE / Module 6 / Slideshow 3 / Researching Health Policy
KEYSTONE / Module 6 / Slideshow 3 / Researching Health Policy
Public Health Foundation of India (PHFI)
 
Looking at implementation: how useful is realist evaluation?
Looking at implementation: how useful is realist evaluation?Looking at implementation: how useful is realist evaluation?
Looking at implementation: how useful is realist evaluation?
valéry ridde
 
Falls prevention initiative npt presentation
Falls prevention initiative npt presentationFalls prevention initiative npt presentation
Falls prevention initiative npt presentation
Normalizationprocess
 
Innovative approaches to researching governance
Innovative approaches to researching governanceInnovative approaches to researching governance
Innovative approaches to researching governance
resyst
 
Where is the “C” in health systems strengthening by Dr Alfonso Rosales, WVUS
Where is the “C” in health systems strengthening by Dr Alfonso Rosales, WVUSWhere is the “C” in health systems strengthening by Dr Alfonso Rosales, WVUS
Where is the “C” in health systems strengthening by Dr Alfonso Rosales, WVUS
achapkenya
 
Project Cycle and Causal Hypothesis _ Theory of Change.pptx
Project Cycle and Causal Hypothesis _ Theory of Change.pptxProject Cycle and Causal Hypothesis _ Theory of Change.pptx
Project Cycle and Causal Hypothesis _ Theory of Change.pptx
GeorgeKabongah2
 
Using the realist approach to evaluate public health policies with a health p...
Using the realist approach to evaluate public health policies with a health p...Using the realist approach to evaluate public health policies with a health p...
Using the realist approach to evaluate public health policies with a health p...
Emilie Robert
 
A toolkit for complex interventions and health technologies using normalizati...
A toolkit for complex interventions and health technologies using normalizati...A toolkit for complex interventions and health technologies using normalizati...
A toolkit for complex interventions and health technologies using normalizati...
Normalizationprocess
 
A Qualitative Methodological Approach Through Field Research
A Qualitative Methodological Approach Through Field ResearchA Qualitative Methodological Approach Through Field Research
A Qualitative Methodological Approach Through Field Research
Diana Oliva
 
Organizing for Sustainability: Including and Engaging Diverse Stakeholders
Organizing for Sustainability: Including and Engaging Diverse Stakeholders Organizing for Sustainability: Including and Engaging Diverse Stakeholders
Organizing for Sustainability: Including and Engaging Diverse Stakeholders
Rahul Mitra
 
manajemen rumah sakit 4
manajemen rumah sakit 4manajemen rumah sakit 4
manajemen rumah sakit 4
rsd kol abundjani
 
Empirical Study of Organizational Citizenship Behavior— Through Knowledge Sha...
Empirical Study of Organizational Citizenship Behavior— Through Knowledge Sha...Empirical Study of Organizational Citizenship Behavior— Through Knowledge Sha...
Empirical Study of Organizational Citizenship Behavior— Through Knowledge Sha...
inventionjournals
 
Shared Governance in Nursing
Shared Governance in NursingShared Governance in Nursing
Shared Governance in Nursing
LORIELENE PARCIA
 
Executive summary:From Evidence to Practice: Addressing the Second Translatio...
Executive summary:From Evidence to Practice: Addressing the Second Translatio...Executive summary:From Evidence to Practice: Addressing the Second Translatio...
Executive summary:From Evidence to Practice: Addressing the Second Translatio...
NEQOS
 
Action research as a methodology that brings together value-based and evidenc...
Action research as a methodology that brings together value-based and evidenc...Action research as a methodology that brings together value-based and evidenc...
Action research as a methodology that brings together value-based and evidenc...
The Tavistock Institute of Human Relations
 
Don de Savigny: Systems Thinking Methodologies for Health Systems Research – ...
Don de Savigny: Systems Thinking Methodologies for Health Systems Research – ...Don de Savigny: Systems Thinking Methodologies for Health Systems Research – ...
Don de Savigny: Systems Thinking Methodologies for Health Systems Research – ...
Rebekah McKay-Smith
 
Doctors 2.0 & YOU - 2017 - Methodological and ethical issues of connected dev...
Doctors 2.0 & YOU - 2017 - Methodological and ethical issues of connected dev...Doctors 2.0 & YOU - 2017 - Methodological and ethical issues of connected dev...
Doctors 2.0 & YOU - 2017 - Methodological and ethical issues of connected dev...
DoctorsTwoPointO And You
 
What is Normalisation Process Theory?
What is Normalisation Process Theory?What is Normalisation Process Theory?
What is Normalisation Process Theory?
NEQOS
 
“Community Pathways to Improved Adolescent Sexual and Reproductive Health” (P...
“Community Pathways to Improved Adolescent Sexual and Reproductive Health” (P...“Community Pathways to Improved Adolescent Sexual and Reproductive Health” (P...
“Community Pathways to Improved Adolescent Sexual and Reproductive Health” (P...
UN Focal Point on Youth, Division for Social Policy and Development
 

Similar to Realist Evaluation : some thoughts about the theoretical foundations (20)

HFG Health Governance Presentation at 2015 USAID Global Health Mini-University
HFG Health Governance Presentation at 2015 USAID Global Health Mini-UniversityHFG Health Governance Presentation at 2015 USAID Global Health Mini-University
HFG Health Governance Presentation at 2015 USAID Global Health Mini-University
 
KEYSTONE / Module 6 / Slideshow 3 / Researching Health Policy
KEYSTONE / Module 6 / Slideshow 3 / Researching Health PolicyKEYSTONE / Module 6 / Slideshow 3 / Researching Health Policy
KEYSTONE / Module 6 / Slideshow 3 / Researching Health Policy
 
Looking at implementation: how useful is realist evaluation?
Looking at implementation: how useful is realist evaluation?Looking at implementation: how useful is realist evaluation?
Looking at implementation: how useful is realist evaluation?
 
Falls prevention initiative npt presentation
Falls prevention initiative npt presentationFalls prevention initiative npt presentation
Falls prevention initiative npt presentation
 
Innovative approaches to researching governance
Innovative approaches to researching governanceInnovative approaches to researching governance
Innovative approaches to researching governance
 
Where is the “C” in health systems strengthening by Dr Alfonso Rosales, WVUS
Where is the “C” in health systems strengthening by Dr Alfonso Rosales, WVUSWhere is the “C” in health systems strengthening by Dr Alfonso Rosales, WVUS
Where is the “C” in health systems strengthening by Dr Alfonso Rosales, WVUS
 
Project Cycle and Causal Hypothesis _ Theory of Change.pptx
Project Cycle and Causal Hypothesis _ Theory of Change.pptxProject Cycle and Causal Hypothesis _ Theory of Change.pptx
Project Cycle and Causal Hypothesis _ Theory of Change.pptx
 
Using the realist approach to evaluate public health policies with a health p...
Using the realist approach to evaluate public health policies with a health p...Using the realist approach to evaluate public health policies with a health p...
Using the realist approach to evaluate public health policies with a health p...
 
A toolkit for complex interventions and health technologies using normalizati...
A toolkit for complex interventions and health technologies using normalizati...A toolkit for complex interventions and health technologies using normalizati...
A toolkit for complex interventions and health technologies using normalizati...
 
A Qualitative Methodological Approach Through Field Research
A Qualitative Methodological Approach Through Field ResearchA Qualitative Methodological Approach Through Field Research
A Qualitative Methodological Approach Through Field Research
 
Organizing for Sustainability: Including and Engaging Diverse Stakeholders
Organizing for Sustainability: Including and Engaging Diverse Stakeholders Organizing for Sustainability: Including and Engaging Diverse Stakeholders
Organizing for Sustainability: Including and Engaging Diverse Stakeholders
 
manajemen rumah sakit 4
manajemen rumah sakit 4manajemen rumah sakit 4
manajemen rumah sakit 4
 
Empirical Study of Organizational Citizenship Behavior— Through Knowledge Sha...
Empirical Study of Organizational Citizenship Behavior— Through Knowledge Sha...Empirical Study of Organizational Citizenship Behavior— Through Knowledge Sha...
Empirical Study of Organizational Citizenship Behavior— Through Knowledge Sha...
 
Shared Governance in Nursing
Shared Governance in NursingShared Governance in Nursing
Shared Governance in Nursing
 
Executive summary:From Evidence to Practice: Addressing the Second Translatio...
Executive summary:From Evidence to Practice: Addressing the Second Translatio...Executive summary:From Evidence to Practice: Addressing the Second Translatio...
Executive summary:From Evidence to Practice: Addressing the Second Translatio...
 
Action research as a methodology that brings together value-based and evidenc...
Action research as a methodology that brings together value-based and evidenc...Action research as a methodology that brings together value-based and evidenc...
Action research as a methodology that brings together value-based and evidenc...
 
Don de Savigny: Systems Thinking Methodologies for Health Systems Research – ...
Don de Savigny: Systems Thinking Methodologies for Health Systems Research – ...Don de Savigny: Systems Thinking Methodologies for Health Systems Research – ...
Don de Savigny: Systems Thinking Methodologies for Health Systems Research – ...
 
Doctors 2.0 & YOU - 2017 - Methodological and ethical issues of connected dev...
Doctors 2.0 & YOU - 2017 - Methodological and ethical issues of connected dev...Doctors 2.0 & YOU - 2017 - Methodological and ethical issues of connected dev...
Doctors 2.0 & YOU - 2017 - Methodological and ethical issues of connected dev...
 
What is Normalisation Process Theory?
What is Normalisation Process Theory?What is Normalisation Process Theory?
What is Normalisation Process Theory?
 
“Community Pathways to Improved Adolescent Sexual and Reproductive Health” (P...
“Community Pathways to Improved Adolescent Sexual and Reproductive Health” (P...“Community Pathways to Improved Adolescent Sexual and Reproductive Health” (P...
“Community Pathways to Improved Adolescent Sexual and Reproductive Health” (P...
 

Realist Evaluation : some thoughts about the theoretical foundations

  • 1. THEORY AND PRACTICE OF REALIST EVALUATION Some thoughts about the theoretical foundations Valéry Ridde, Emilie Robert, Anthony Lacouture July, 2; 2015 WHO Meeting : Realist evaluation of national strategic planning and policy processes equitesante.org
  • 3. 3
  • 4. RESEARCH QUESTION • WHAT ? Equitesante.org 4 • WE ALREADY KNOW WHAT WORKS…
  • 5. FROM WHAT TO…. HOW Equitesante.org 5 FOR WHOM WHY TO WHAT EXTENT IN WHAT CIRCUMSTANCES
  • 6. PUBLIC HEALTH AND COMPLEXITY « Multiple entangled actions, acting in often non-linear relationships, organized in a specific context and understood as open systems, researcher who observe in fact itself part of it » Blaise et al. 2010 Equitesante.org 6
  • 7. 7 CHARACTERISTICS OF COMPLEX INTERVENTIONS Volitions Equitesante.org 7 Implementation Contextes Time Outcomes Rivalry Emergence Pawson, 2013
  • 8. CRITICAL REALISM Equitesante.org 8 POSITIVISM CRITICAL REALISM RELATIVISM QUESTIONS Is the policy or intervention (cost)-effective? What works for whom under which conditions? How do actors experiences, power and understand? DISCIPLINES Epidemiology Welfare economics Political science Policy analysis Organizational studies Socio-anthropology DESIGN Deductive Fixes Abductive Pragmatic/MM Inductive Flexibles Gilson, 2012; Robson, 2002
  • 9. CRITICAL REALISM • Reality exists outside the human constructions, but it is perceptible only through our senses Equitesante.org 9 Maxwell, Pawson, Bhaskar, etc.
  • 10. GENERATIVE CAUSALITY • We are not looking for immutable law vs sucessionist notion of causality • Importance of mechanisms/context vs intervention • But regularities (semi-reg : Lawson) – semi-predictable patterns – pathway of program functioning • Patterns = CMO Equitesante.org 10 (Pawson, 2006)
  • 12. MECHANISM DEFINITION • is hidden but real, sensitive to the context and produces outcomes • is an element of reasoning and reactions of agents in regard of the resources available in a given context to bring about changes • evolves within an open space-time and social system of relationships Equitesante.org 12 Lacouture et al, 2015
  • 13. MECHANISM • Elements of reasoning of agents and their choices about how the change will be achieved through the implemented intervention • Reactions of agents in response to the resources provided by the intervention 13 Equitesante.org • Pre-existing characteristics of the individuals, localities, situations or systems in which an intervention is being set up (= CONTEXT) • Intentional measures, activities or a rational plan taken by program implementers (= STRATEGIES OF INTERVENTION) Lacouture et al, 2015
  • 15. NOT A MECHANISM Equitesante.org 15 Actor’s attitudes : health workers’ motivation and satisfaction (A1), the provider–patient relationship (A2), patients’ satisfaction (A3), patients’ perceptions on health and on financial access to healthcare (A4), and health workers’ coping strategies (A5). Ridde et al, 2012
  • 16. MECHANISM EXAMPLE CMOc of PR synergy related to interpreting and disseminating research findings (strategy). •As a result of the long-term collaborative relationships between academic and community stakeholders (context), locally informed and accessible scientific study findings were presented to the community (strategy). The audiences at the presentations felt comfortable and safe participating in the context of their community ownership of the project (mechanism) and made sense of the data in their own terms. As a result, new explanatory insights into the data were generated (outcome). Equitesante.org 16 Jagosh et al, 2014
  • 17. THEORY-DRIVEN EVALUATION • Program/intervention theory (Weiss, Chen, etc.. ) • Causal model • Middle range theory (initial and final) Equitesante.org 17
  • 18. INTERVENTION THEORY • "A program is a theory and an evaluation is its test" (Rein, 1981) • Programs are “theories incarnate”. (Pawson 2012) • “It is a small theory attempting explanation of specific treatment processes, not a large theory of general social phenomena” (Lipsey, 1993) Equitesante.org 18
  • 19. 19
  • 20. INITIAL PT IN UGANDA • Management practices and leadership styles (interven- tion) that are supportive of autonomy lead to positive work behaviour of volunteers, such as improved task per- formance and persistence (outcomes), because they satisfy the psychological needs of autonomy, competence and relatedness (mechanism). More specifically, such management and leadership contribute to a shift from ‘external motivation’ towards ‘internalised motivation’ for the volunteers whose initial engagement was more driven by external motives. Internalised motivation leads not only to positive work behaviour, but also to well- being. Such management and leadership furthermore allow volunteers whose initial engagement was mainly driven by internal motives to remain engaged and perform well. Equitesante.org 20 Vareilles et al, 2015
  • 21. MIDLE RANGE THEORY • « Middle-range theory involves abstraction, of course, but they are close enough to observed data to be incorporated in propositions that permit empirical testing. » Merton, 1967 • Level of abstraction = useful + testable/data Equitesante.org 21
  • 22. INITIALE MIDLE RANGE THEORY • "Hospital managers of well-performing hospitals deploy organisational structures that allow decentralisation and self-managed teams and stimulate delegation of decision-making, good flows of information and transparency. Their HRM bundles combine employment security, adequate compensation and training. This results in strong organisational commitment and trust. Conditions include competent leaders with an explicit vision, relatively large decision-making spaces and adequate resources." Equitesante.org 22 Marchal et al, 2010
  • 23. INITIALE MIDLE RANGE THEORY • "‘A pluralistic health system harbours a web of accountability relationships between actors who combine the roles of account-holder and accountor.  • Public accountability is actualised when actors are answerable to the public and remedial action is undertaken. Public accountability requires both answerability and enforceability in order to be actualised. The answerability or the capability of the DHMT, SRH INGOs and partnerships to inform, evaluate and report in an open manner requires transparency and clarity on whom they represent and deliver services to. Answerability is actualised through practices grounded in compliance and persuasion. • Enforceability is grounded in the capability of the public to demand accountability on the one hand and in meta-governance, i.e. the function, exercised by a state actor(s), of regulating, monitoring and sanctioning on the public’s behalf, on the other hand. • Accountability practices operate along four dimensions (social, political, organisational and the provider dimension). Each dimension has specific bundles of strategies, practices, relationships and outcomes. Accountability is embedded in vertical, horizontal and partnership governance arrangements. • Multi-level governance arrangements weaken public accountability when there is confusion over roles and responsibilities between governing actors.’" Equitesante.org 23 Van Belle, 2014
  • 24. CMO : HEURISTIC FRAMEWORK Ugly circumlocution (Pawson et al, 2012) Equitesante.org 24Robert & Ridde, 2014
  • 25. Original CMO for the CHW • Context – Lack of community awareness regarding safe motherhood, danger signs during pregnancy, labour, and postpartum periods and where to go for services. • Programme response/mechanism – Training and deploying trusted members of the community instills ‘knowledge’ of dangersigns during pregnancy and labour, safe delivery practices, as well as newborn care. Regular visits from community members build relationships between women and health workers, leading to ‘trust’ in the health system. • Outcome – More women aware of complications; increased care seeking. Equitesante.org 25Adams et al, 2015
  • 28. CMO CONFIGURATION A community experiences a high level of unemployment to which an employment training program is offered in a remote part of town (context). But the program has low enrollment and high attrition, and few people are trained (outcome). The reason is that people feel disillusioned by the lack of effort by program planners to ensure adequate public transportation to the venue (mechanism). Equitesante.org 28 Jagosh et al, 2013
  • 29. Initiale CMO CONFIGURATION Participants who are motivated and share a good rapport’ (context) may ‘share ideas, experiences and support one another’ (mechanism), which may be associated with a smoking cessation and improved weight management (outcome) Equitesante.org 29 Mackenzieet al, 2009
  • 30. THEORIES AND CMO volunteer motivation & work behaviour • Crowding theory • Intrinsic motivation theory • self-determination theory • Gift exchange and high- performance HRM theory • Consistency theory • Organisational commitment • Social exchange theorY participatory research assessment 30 Equitesante.org Vareilles et al, 2015 Jagosh et al, 2013 • Theory of Capacity Building • Communicative Action Theory • Empowerment/emancipati on • Collaborative Management • Partnership Synergy Theory
  • 31. FROM CMOi TO CMOf Equitesante.org 31 Pawson et al, 2012
  • 32. 32
  • 33. 33
  • 34. 34
  • 36. MECHANISMS CHALLENGES Equitesante.org 36 • a sense of self efficacy, • an increase in self esteem, • assurance that there is a system for back-up support
  • 40. FROM CMO TO MRT Equitesante.org 40 Abstraction Empirical MRT1 THEORY1 C1 M1 O1 PROGRAM THEORY C2 M2 O2 C3 M3 O3 C4 M4 O4 C1 M1 O1 C1 M2 O3 C1 M1 O3 C2 M2 O1 C1 M3 O3 C1 M4 O3 MRT2 MRT3 MRT4 MRT5 THEORY2 THEORY3 STARTSTART
  • 41. FROM CMO TO MRT Equitesante.org 41 Abstraction Empirical MRT1 THEORY1 C1 M1 O1 PROGRAM THEORY C2 M2 O2 C3 M3 O3 C4 M4 O4 C1 M1 O1 C1 M2 O3 C1 M1 O3 C2 M2 O1 C1 M3 O3 C1 M4 O3 MRT2 MRT3 MRT4 MRT5 THEORY2 THEORY3 STARTSTART

Editor's Notes

  1. Complex social interventions …... ... are theories or consist of several theories. … ... involve the participation of numerous stakeholders. .. ... consist of a chain of decision processes. .. ... are not linear and are subject to feedback loops. .. ... are entrenched in several social systems and several contexts. … ... are permeable to the influence of other interventions. …. .. are open learning systems. ...
  2. UN DE DEUX ???? J,aime bien celui de droite car il mets en avant la question, mais on peut adapter
  3. La RE est fondamentalement une theory driven evaluation
  4. La RE est fondamentalement une theory driven evaluation
  5. A mechanism is hidden but real: Existing prior to the intervention, but latent, a mechanism reveals itself during implementation of the intervention within a given context. Sensitive to the context, it produces outcomes. A mechanism is an element of reasoning and reactions of agent(s) in regard of their ability to use the resources available in a given context to bring about changes : A mechanism results of the interaction between agents, intervention and structures. It reflects the logic of intervention of the various actors involved directly (e.g. stakeholders) or indirectly (e.g. patients) in the intervention. A mechanism evolves within an open space-time and social system of relationships : A mechanism is dynamic and it may be interacting with other mechanisms (family of mechanisms), with elements of context or with the effects it has itself produced (i.e. positive or negative feedback).
  6. A mechanism is hidden but real: Existing prior to the intervention, but latent, a mechanism reveals itself during implementation of the intervention within a given context. Sensitive to the context, it produces outcomes. A mechanism is an element of reasoning and reactions of agent(s) in regard of their ability to use the resources available in a given context to bring about changes : A mechanism results of the interaction between agents, intervention and structures. It reflects the logic of intervention of the various actors involved directly (e.g. stakeholders) or indirectly (e.g. patients) in the intervention. A mechanism evolves within an open space-time and social system of relationships : A mechanism is dynamic and it may be interacting with other mechanisms (family of mechanisms), with elements of context or with the effects it has itself produced (i.e. positive or negative feedback).
  7. La RE est fondamentalement une theory driven evaluation Théorie de programme : théorie descriptive de la manière dont l'intervention est censée produire ses effets, et qui inclut nécessairement (pour l'évaluation réaliste) le modèle causal. Ce modèle causal s'appuie sur des théories ou des modèles conceptuels qui constituent le point de départ de l'évaluation réaliste. C'est là où il y a confusion avec la TMP, à mon avis. C'est l'étape n°1 de l'évaluation réaliste. - Théorie de moyenne portée : théorie explicative de la manière dont le contexte influence les mécanismes mis en oeuvre par l'intervention pour produire des effets. C'est le produit final de l'évaluation réaliste.
  8. La RE est fondamentalement une theory driven evaluation
  9. Deux façons de présenter des théories de programme mais cela n,est pas suffisant, l’intérêt de la RE est d’aller en détail, notamment pour les mécanismes et les CMO ER : l’intérêt de la RE est surtout de faire le lien entre contexte et mécanisme(s) d’une intervention, ce que ne peut pas faire un théorie de programme.
  10. No all good meca = attention !!
  11. DEUX DIAPOS POUR MONTRER COMMENT NE PAS FAIRE ER : ce n’est pas un bon exemple : à enlever ! Il faut – je crois – éviter de montrer des tableaux de liste parce que cela ne montre pas les liens de causalité, et laisse penser aux gens que c’est suffisant d’en arriver là. En plus la plupart des mécanismes n’en sont pas.
  12. DEUX DIAPOS POUR MONTRER COMMENT NE PAS FAIRE ER : ce n’est pas un bon exemple : à enlever ! Il faut – je crois – éviter de montrer des tableaux de liste parce que cela ne montre pas les liens de causalité, et laisse penser aux gens que c’est suffisant d’en arriver là. En plus la plupart des mécanismes n’en sont pas.
  13. Je mettrais bien ici un de tes exemples de thèse, car cela sera un seul, aucun risque, et proche de ce que les gens connaissent ?
  14. Je mettrais bien ici un de tes exemples de thèse, car cela sera un seul, aucun risque, et proche de ce que les gens connaissent ? ER : tu peux mettre celui-là ; Les défaillances dans la mise en œuvre des politiques d’exemption des paiements directs et les dysfonctionnements parfois préexistants du système de santé sapent les relations entre usagers et prestataires, et contribuent à faire émerger un sentiment de défiance des usagers envers les prestataires de soins, voire envers le système de santé. Cela favorise le phénomène de contournement de certaines formations sanitaires ou le choix de prestataires privés ou de soins domestiques.
  15. Comment formuler des CMO
  16. Revoir, adapté ? ER : je me rappelle pas trop ce que cela veut dire… Pas sûre que ce soit utile.
  17. L,importance du contexte et dit par tous, pas seulement Pawson et son école
  18. ER : pas certaine de ce schéma… Ca laisse penser que le concept de « contexte » est fourre-tout. Or il faut insister sur le fait que n’est contexte qu’un élément qui déclenche un mécanisme pour produire un effet. C’est la clé – à mon avis – de la compréhension de l’approche réaliste. Il faut que le public soit capable de comprendre que l’enchaînement contexte-mécanisme-effet peut se décliner à différents niveaux d’analyse, et différents niveaux d’abstraction. La définition LA plus importante est celle de mécanisme. Le fondement Le plus important est le réalisme critique, c’est-à-dire la relation entre contexte-mécanisme-effet et les semi-régularités. Je crois que l’on devrait commencer par là, avant de parler des théories.
  19. ER : pas certaine que ce soit pertinent.
  20. Défis de disposer d’outcomes
  21. Défis de disposer d’outcomes
  22. Processus de construction des CMO ER : il s’agit ici du processus de réflexion et d’analyse, pas nécessairement de construction des C-M-O (car cela concerne également la TMP). J’ai changé la diapo de place parce que je pense qu’on devrait présenter le processus avant le niveau d’abstraction.
  23. PT => CMO => MRT => ER : à la place d’études empiriques, tu peux avoir théorie du programme, qui est plus collée à la réalité, sauf que la théorie du programme explicite la manière dont les concepteurs ont pensé que fonctionner l’intervention, et ne représente pas nécessairement ce qui se passe dans la réalité. C’est là toute la nuance ! De plus, la théorie du programme n’est pas explicative mais descriptive, et ne peut donc pas mettre en évidence les interactions avec le contexte.