Anne LaFond presented as part of a panel at the 2015 Evaluation Conference on using case-based methods for evaluating complexity in the health sector, sharing insights from various JSI case studies.

1. Using case-based
methods for
evaluating complexity
in the health sector
Anne LaFond
John Snow, Inc. (JSI)
American Evaluation Association
November 2015
Chicago, Illinois

2. ​Overview
Why focus on case
studies?
What are case studies?
Illustrations
Overview

3. ​Overview
• Complexity: health
systems and health
interventions
• The setting matters
• Greater depth of
understanding needed
The Context
Going beyond simple
solutions
Replicating success
Going to scale

4. ​Overview
• Research method or an
empirical inquiry that
investigates a contemporary
phenomenon within its real-life
context
• When the boundaries between
phenomenon and context are
not clearly evident, and
• In which multiple sources of
evidence are used.
The Case
Study
The case as an
example for learning
about interventions in
context
Mixed methods
Comparative cases to
increase potential
learning and relevance

5. ​Overview
Case study research
comprises an all
encompassing method –
covering the logic of design,
data collection techniques,
and specific approaches to
data analysis
The Method
More variables than
data points.
Relies on multiple
sources of evidence,
requiring the data to
converge or triangulate.
Theoretical propositions
to guide data collection
and analysis.

6. 6
Not…..

7. 7
Case-based research is

9. Lessons from the 2013 IOM
Evaluation of PEPFAR:
Use of Case-Based
Approaches
Bridget Kelly

10. WHAT DRIVES ROUTINE
IMMUNIZATION SYSTEM
PERFORMANCE IN AFRICA?
A MIXED METHODS COMPARATIVE CASE STUDY
APPROACH
THE AFRICA ROUTINE IMMUNIZATION SYSTEM
ESSENTIALS PROJECT (ARISE)

11. ​Overview
• History of investment in
national immunization
programs in Africa.
• Why does coverage
improve in some settings
and not in others?
Context
The Complexity:
Many variables
associated with high
levels of
immunization
coverage but little
understanding of the
dynamics of
coverage
improvement.

12. Conceptual Framework: Causal Loop
Diagram for Demand for Immunization
Dynamics
Rwashana A, Williams D, Neema S. Systems dynamics approach to immunizationhealthcare issues in developing
countries: a case study of Uganda, Health informatics J 2009.

13. Penta3 Coverage for All Districts
(2008 and 2010, Ghana)

14. ​Overview
• Mixed methods, multiple case
studies
• Case selection: Countries with
DPT3 coverage improvement
• Nested case: Three districts per
country with recent DPT3
coverage improvement and one
district with steady or
unchanged coverage
Methods

15. ​Overview
• Focus on district level and the dynamics of
change
• Mixed methods: Coverage trend analysis
and verification; RI Situation Analysis
(service readiness); Open-ended, semi
structured key informant interviews
district, facility, community levels; FGD;
field observation.
• Iterative field level analysis by district
• Grounded theory building
• Interdisciplinary team: Routine
immunization expert, researchers, country
program
• Cross district case consistency
• Close collaboration with technical and
country experts
• Country level validation (stakeholder
workshop)
Design &
Data
Collection

16. Theory building

17. Cadre of
Community
Centered
Health
Workers
Health
System and
Community
Partnership
Regular
Review of
Program and
Health
Worker
Performance
Immunization
Services
Tailored to
Community
Needs
Four Direct Performance Drivers
LaFond, et.al, 2014

18. Take
vaccination
into heart of
the community
More workers,
build trust, local
support, vaccine
supply.
Raised
awareness,
improved
access,
increased use
Cadre of Community-centered Health Workers
Effect
Mechanism
Transformational
step

19. Ethiopia
“75% of Health Extension
Workers’ (HEW) time is spent
on house-to house to do
sensitization and deliver
services. As a result of this
effort the awareness of the
community about the
importance of immunization
has improved. This motivates
HEWs to become more
dedicated to their work.”

20. “Community Health Nurses are
also very much in touch with the
community. For instance the new
outreach site being introduced
was at the request of the
community members and the
Community Health Nurses are in
the process of discussing with
them the most convenient day
for the Child Welfare Clinics in
their community.”

21. 21
Challenges
• Quality and availability of
coverage data to select
cases
• Consistent approach
across country cases
• Intensive fieldwork stage
with iterative analysis and
theory building

22. What did we learn from use of case-based
approaches?
• Appropriate and relevant for learning about
and within complexity
• Expands and enriches the tools available
beyond indicators and metrics
• Lends credibility through techniques to help
understand “how” and “why” questions.
• Triangulation is critical for analysis and
interpretation

23. Guidance
• No need to be intimidated or
apologetic
• What does “best practice” mean in
this context?
• Fit-for-purpose
• Informed and intentional
• Well-documented

24. information
​For more information,
please contact:
Anne_LaFond@jsi.com
thank you
ARISE was funded by
the Bill & Melinda Gates Foundation
and implemented by
Anne LaFond, Natasha Kanagat, Jenny
Sequeira, Robert Steinglass, Rebecca Fields
(JSI). Judith Justice (Ethiopia), Cheikh Niang,
(Cameroon) and Ann Larson (Ghana)
in collaboration with
Sangeeta Mookherji, George Washington
University

25. FormoreinformationaboutJSI’s
research,monitoring,evaluation&
healthinformationwork,contact
chime@jsi.com
Follow
@jsihealth