This presentation by David Peters of the Future Health Systems Consortium was given at the Global Symposium on Health Systems Research in November 2010.

1. 1
Objectives and Evidence
November 18, 2010
Evidence on Improving Health
Service Delivery in Developing
Countries

2. Session Objectives
 Understand evidence of different strategies
to improve delivery of health services
 Recognize strengths and limitations of
methods used for “evidence-based public
health”, implications for future research
and decision-making
 Identify how institutions and incentives
influence selection, implementation, and
results of improvement strategies
2

3. Key Questions
 Which strategies best improve the delivery
of a range of health services in a given
context?
 Which factors help or hinder implementation?
 Which strategies will ensure that the poor
and disadvantaged will benefit?

4. The Conventional Pathway For
Improving Health
1.Choose the right (cost-effective) health
interventions
2.Set ambitious, common targets
3.Fund them adequately
4.Implement interventions as designed

5. Do Countries Follow a Common Pathway to
Reducing Childhood Mortality According to
Millennium Development Goals?
Matsubayashi & Peters (under review)
Multi-level models, each line represents one country

6. Purpose
•Health Status
(MDGs)
•Financial
protection
•Trust in health
system
GOVERNMENT
Governance
(Leadership; Stewardship; Planning;
Legislation; Regulation)
Financing
(Revenue; Pooling; Allocation; Payment)
Information
(Disclosure; Research; Evaluation)
Health
Services
•Access
•Coverage
•Utilization
•Efficiency
•Quality
PROVIDER
ORGANIZATIONS
Management Systems
& Inputs
Human Resources;
Financial Management;
Information Systems;
Drugs and Medical
Supplies (Logistics);
Medical Technology;
Maintenance &
Buildings;
Auxiliary Services
PEOPLE
Poor /
Non-Poor
Communities
Advocacy
groups
Consumer
protection
groups
MACRO ENVIRONMENT
HEALTH SECTOR ENVIRONMENT
Enabling & Inhibiting
Enabling & Inhibiting
Factors
Factors
STRATEGY
to strengthen health services
Health Services Strategies Analytic
Framework

7. 7
Knowing What vs. How vs. Why
 What works
 “Evidence” for simple interventions
 How does it work
 Knowledge of context, actors, &
implementation processes for complex
change
 Why do it
 Beliefs and values for motivated and
sustained action

8. 8
Thinkers on evidence for causation who
didn’t need epidemiology or economics
Confucius Aristotle
Sutra of Buddha
Adi Shankara Ibn Rushd
JS Mills
David Hume

9. 9
“Parachutes reduce the risk of injury after
gravitational challenge, but their effectiveness has
not been proved with randomized controlled trials”
Smith & Pell (2003) BMJ

10. 10
What Do We Want to Know? Questions,
Inferences, How To Investigate
Primary question Inference Design Implications
Is any measured health effect
due to the implemented
strategy?
Probability Controlled (cluster randomized) trials;
health strategy implemented in some
areas and not in others
Is any measured health effect
likely due to the strategy rather
than other influences?
Plausibility Concurrent, non randomized clusters;
strategy implemented in some areas and
not in others; before-after or cross-
sectional study in program recipients and
non-recipients
Are health indicators changing
among beneficiaries of a
strategy?
Adequacy Before-after or time-series in program
recipients only
How did implementation of the
strategy lead to effects on health
behavior, services or status?
Explanatory Mixed methods: Repeated measures of
context, actors, depth and breadth of
implementation across subunits +
Qualitative methods: key informant
interviews, focus groups, historical
reviews, and triangulation of data sources

11. 11
Who wants to know?
Which question is most important to:
 Policy-makers
 Managers
 The public
 Development partners
 Funding agencies
 Researchers

12. Systematic Reviews
Chapter Main issues addressed
1. Strategies to strengthen
health services
What is the evidence of the effect of different
approaches to strategies to strengthen health
services? What are the main enabling and inhibiting
factors influencing their effect?
2. Community empowerment
strategies for health
What is the evidence of the effect on health services
or outcomes of different strategies to empower
communities?
3. Strategies to improve health
care provider performance
What is the evidence of the effect on the performance
of health workers of different strategies to improve that
performance?
4. Strategies to strengthen the
performance of health
organizations
What is the evidence of the effect of different
strategies to improve the performance of health
services organizations?

13. Studies on How Health Services
Change Occurs
Chapter Main issues addressed Methodology
5. Cross-country
changes in health
services
What are the trends over time of specific health
service outputs across LMICs? How do changes in
one priority health service affect changes in other
health services? What are the contextual factors
that most influence health service outputs at
national level?
Longitudinal statistical analysis
of cross-country data on health
services and contextual factors
6. Evaluation of
changes in health
services, financing, and
health status in World
Bank-assisted health
projects
How well do World Bank-assisted project
evaluations measure changes in health services,
financing, and health status? What are the main
factors associated with measuring changes?
Documentation analysis using
standardized definitions and
cross-sectional statistical
analysis
7. Institutional context
of health services
What are the main institutional factors influencing
the delivery of health services in LMICs?
Literature review
8. Seven country case
studies
Describe the strategies of significant scale for
strengthening health services delivery in the real-life
context in which they occurred. What are the
presumed causal links between the strategies and
their effects? What are the lessons learned?
Mixed methods case studies
(data and documentation
review, key informant
interviews)

14. Some Systematic Reviews on What
Works
Systematic review Number of
studies
screened
Number of
studies with
adequate designs
1. Health services strengthening
strategies in LMICs
208 150
2. Community empowerment in
health strategies in LMICs
27,000 156
3. Improving health worker
performance in LMICs
>105,000 223 (ongoing)
4. Improving the performance of
health services organizations in
LMICs
127,000 88
14

15. Strategies to Improve Implementation
of Health Services
Strategy Type Examples
Public Oversight Policy Reviews; Regulations; Contracting;
Supporting Consumer Organizations
Provider Based Quality Improvement Programs; Training;
Human Resource Management; Logistics
Systems; Decentralization of Public Services
Community Based Community Development Programs;
Demand Creation
Health Financing Provider Payment System Change; Subsidies;
Health Insurance
1
5

16. Study Details
16

17. Conclusions & A Way Forward
17

18. Some Over-Arching Findings
 Implementation faults are very common
 Many different types of strategies can succeed, but are
not replicable in much detail – Strategies with the same
label do very different things
 Institutions involved and how implementation occurs
matter greatly
 Policy makers define strategies, but often have limited
influence on how they are implemented
 Strategies produce many unintended consequences, not
predictable in detail
 Not nearly enough attention has been paid to
demonstrating how to improve services for the poor
18

19. How Strategies are Implemented Matters:
Odds of Success from Systematic Review
Randomized
Controlled
Trials
All
“Adequate”
Studies
Odds ratio Odds ratio
Community coordination and organization
.. 4.6**
Local adaptation of the intervention 9.3 4.3 *
Broad-based support of various
stakeholders
.. 3.9 *
Consultation and engagement of powerful
interest groups
2.8 3.8**
Flexibility and modification through
stakeholder feedback
.. 3.4 *
Representation from powerful interest
groups
2.4 3.0 *
Constraints reduction plans 6.7 2.7 *
* Pvalue<0.05; ** Pvalue<0.01 1
9

20. Practical Lessons from
Implementation Experience
 There is NO BLUEPRINT, NO MAGIC BULLET for
successful implementation
 How a strategy is implemented is at least as important
as what strategy is pursued
 Improvement for the poor involves planning for
benefits to reach the poor, ensuring regular
measurement of impact on the poor, and providing
oversight to ensure the poor benefit
 A “learning and doing” approach underlies successful
implementation of many different strategies
2
0

21. What “Learning & Doing” Means
 Asking difficult questions about leaders,
laggards, and lessons
 Using information intelligently:
 intended results,
 unintended consequences
 explanations, connections
 Involving key stakeholders and institutions
 Revise implementation plans, authorities,
accountabilities, incentives as you learn
21

22. Structured Learning & Doing
Approaches
Customers &
Beneficiaries
Provider
Organizations
Institutional Support
Public Participation Iterative Learning Flexible, Capable
Management
Strategies
22

23. A Bright Future for Health Systems
 Many strategies work
 Use data and disclose information
 Involve all key stakeholders
 Focus on needs of disadvantaged
 Use opportunities for learning
23