A "deep dive" learning session delivered at the 2015 Accord Summit on Excellence in Water, Sanitation & Hygiene (WASH). More details: http://wash.accordnetwork.org

1. Deep Dive:
Measuring & Improving the
Lasting Benefit of WASH Services
Jonathan Mitchell, Concentric Development
Jonathan Wiles, Living Water International

2. 1
WHAT to Measure?

3. Survey of Past Project Functionality
Living Water International
Program Excellence Group
June 6, 2013
» “How many Living Water-
implemented water systems
are still working today?”

4. All with GPS = 10,466

5. With GPS in Current Operational Areas = 9,435

6. System installations with GPS in Current Operational Areas = 5,717

7. System installations with GPS in Current Operational
Areas not experiencing violent conflict or pandemic
= 5,213

8. Table Discussion 1
Question:
» What is YOUR “learning lab,” and what
question do YOU want to answer about the
lasting benefits of WASH services?
10 minutes

9. 2
HOW to Measure?

10. System installations with GPS in Current Operational
Areas not experiencing violent conflict or pandemic
= 5,213

11. Calculating Samples
Country Population Sample
India 1332 100
Ethiopia 140 62
Ghana 54 37
Kenya 81 91
Rwanda 376 84
Uganda 290 69
Zambia 109 42
El Salvador 475 81
Guatemala 290 79
Haiti 109 58
Mexico 125 57
Nicaragua 288 77
837
» Attrition: Burkina Faso, Peru,
Honduras, Zimbabwe.
» Random samples were
calculated at home office
& assigned to country
operations.
» Samples were calculated for
a 10% margin of error & 95%
confidence interval at country
level (yielding 2% global MoE).

12. Designing the Survey Logic
Begin
Choose the site
from your drop-
down menu
Do you observe any
problems with the WP?
q Limited flow
q Poor siting
q Potential contamination
q Objectionable taste/smell
q Other: ___________
YES
What is the water source?
q Protected spring/gravity
q Borehole/gravity
q Other (specify):___
What type of hand pump?
q AfriDev
q India Mark II
q Other (specify):___
What
type of
water
point
is it?
Tap/standpipe
Hand pump NO
Click here to capture geo-
tagged photo of water point
being operated (flowing
water visible if functional)
Is any
water
currently
flowing?

13. No
water
Water
How much water does the WP
produce?
q Enough
q Insufficient
q Inconsistent/seasonal
Do local users observe any
problems with the WP?
q Limited flow
q Poor siting
q Potential contamination
q Objectionable taste/smell
q Other: ___________
Is this WP a primary drinking
water source for local users?
When was the WP last
functional?
What problems contribute to
its non-functionality?
q No/poor supply chain
q No technical support
q Insufficient management
q No user demand/motivation
q Poor siting
q Other:___________
If/when this WP is
functioning, would it be a
primary drinking water
source for local users?
Who is responsible for
managing the WP?
q Village Water Committee
q Institution (school, clinic…)
q Local government
q District/county government
q Other: ___________
Do local users pay for the
use/operation/maintenance
of the WP?
q Per container
q Monthly fee
q Annual fee
q When system breaks down
q Never pay
q Other:__________
Do you have any final
observations or comments?
Thank
you!

14. Choosing Field Methods
» Balancing Credibility & Cost (Internal vs. external data collectors)
» Electronic vs. paper
» GPS validation
» Multi-language support (English, Creole, Amharic, Spanish, Swahili)

15. Managing Field Costs
Country Sample “Grant”
India 100 $4,500
Ethiopia 62 $2,000
Ghana 37 $1,500
Kenya 91 $2,500
Rwanda 84 $2,500
Uganda 69 $2,000
Zambia 42 $1,500
El Salvador 81 $2,000
Guatemala 79 $2,000
Haiti 58 $2,000
Mexico 57 $2,000
Nicaragua 77 $2,000
837 $26,500
» Internal “Grant” logic
» 31-50: $1,500
» 51-70: $2,000
» 70+: $2,500

16. Collecting the Data
Country Population Sample Collected
India 1332 100 94
Ethiopia 140 62 62
Ghana 54 37 32
Kenya 81 91 84
Rwanda 376 84 84
Uganda 290 69 66
Zambia 109 42 39
El Salvador 475 81 80
Guatemala 290 79 79
Haiti 109 58 53
Mexico 125 57 49
Nicaragua 288 77 70
837 792
(95.7%)

18. Table Discussion 2
Question:
» What kind of survey or study would help YOU
answer YOUR question about the lasting
benefit of WASH services?
15 minutes

19. 3
WHY to Measure (and Improve)?

20. Analyzing the Data: Main Findings
Functional (77%)
Non-functional (17%)
Limited functionality (6%)
• Representative sample of Living Water systems over 15 years (1999-2013)
• Study was carried out in Q4 2014, including 792 water points in 12 countries
• 95% confidence level, 2% +/- margin of error
• Mechanical functionality only

21. Analyzing the Data: Regional Variances
88% 67% 78%
Latin America
& Caribbean Africa India

22. Analyzing the Data: Technology Choice
77%
Protected
Spring
Hand
Pump
Borehole
/Piped
Rainwater
Harvesting
78% 78% 75%

23. Financial Contributions
No Payment Scheme With Payment Scheme
76%56%
Africa Region

24. Community Management
0
50
100
150
200
250
Committee Institution
75%
0
10
20
30
40
50
60
Committee Institution
No or Low
Functionality
Functional
56% 85%70%
Africa India

25. Why do water systems fail?
» Recurring themes:
» Poor siting or low demand (possibly indicating poor
pre-implementation community mobilization)
» Poor or under-equipped community management
» Poor technical support and supply chain

26. Impact on Organization
Pursue
Excellence
Makes sector knowledge
“real” to organization
Builds knowledge &
will to improve
program practices
Builds culture of
measurement &
learning at all levels
Creates a sense of
urgency to improve
policies & systems

27. Table Discussion 3
Question:
» If YOU could accurately answer YOUR
question about the lasting benefit of WASH
services, what would the implications be for
the public, your board, your donors, and
(especially) your program implementation?
10 minutes

28. 0%
10% 10%
40%
31% 31%
23%
8% 8%
17% 17%
9%
22%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Pastor lives in
village
In last week > 1 week > 1 month > 6 months
Time since last pastor visit
Bihar
Jharkhand
Total

29. 0% 0% 4% 5%
20% 20% 18%
39% 40%
25%
75%
20%
17%
28%
21%
11%
60%
44%
0%
20%
40%
60%
80%
100%
120%
1 2 3 4 5 6 7 8 9 11
Well Age Years
Repair Needs By Well Age
Minor Repair
Major Repair
41% of wells
(80) were in
need of repair

30. 2015 SUMMIT on
EXCELLENCE IN WATER,
SANITATION & HYGIENE