Tata AIG General Insurance Company - Insurer Innovation Award 2024
Why different methods generate different numbers: Case study from Ethiopia
1. Why different methods generate different
numbers: case study from Ethiopia
John Butterworth, IRC International Water and Sanitation Centre
2.
3. Key points
• Provider (scheme inventory) and user (household survey) data
are, obviously, fundamentally different
• Ministry of Water and Energy collect one, Central Statistical Agency the
other, but the National WASH Inventory is an exception
• Neither method good or bad, but both have strengths and weaknesses
• Neither are well related to the published targets
• Parallel monitoring processes at global, national, and local levels likely
to persist: may even be more complex post-2015
• Investment in multi-stakeholder reconciliation processes vital
5. Focus on JMP estimates
• Data, data, data:
– the most recent data hasn’t always been available and trend
lines are sensitive
– accuracy and representativeness of underlying household
survey data
• Definition of ‘improved water facilities’
• Errors
• Transparency in methodology and calculations
6. Focus on MoWE estimates pre-2010
• Estimates based on design capacity: outputs rather than outcomes
• Occasional inventories, new schemes added
• Simple, reliable, low cost: provides useful data for frontline service
delivery
• Multiplication factors the Achilles heel: under- and over-estimates
• Overestimates generated by inclusion of non-functional
schemes, clustering
7. Post-2010 and the National WASH Inventory
• Standard nationwide survey
• Uniquely collected provider and user data
• Beyond water points and households: institutions included
• Estimate made of users within quantity/distance norms and total
users
• Data approved, to be validated, made available and put to use
8. Where are we? Somewhere in the 40s
• JMP (2011) – 39%
• DHS(2011) – 42%
• MoWE NWI household survey – 45%
• MoWE NWI scheme inventory – 49%
• Bearing in mind water quality issues and risks of slippage, the rural
water supply challenge is bigger than previously acknowledged
9. Top priorities: what is most important?
• Transparency: methods, access to data
• Put data to use
• Update and maintain the NWI
• Strengthen links with Central Statistical Agency
• Promote monitoring as a research topic
• Investing in communications and multi-stakeholder reconciliation
processes
10. Key points
• Provider (scheme inventory) and user (household survey) data
are, obviously, fundamentally different
• Ministry of Water and Energy collect one, Central Statistical Agency the
other, but the National WASH Inventory is an exception
• Neither method good or bad, but both have strengths and weaknesses
• Neither are well related to the published targets
• Parallel monitoring processes at global, national, and local levels likely
to persist: may even be more complex post-2015
• Investment in multi-stakeholder reconciliation processes vital
Editor's Notes
Acknowledgements: 2 previous seminars during the first phase of RiPPLE (2010 and 2011), a DFID supported research partnership. The MoWE and the NWI team.
Acknowledgements: 2 previous seminars during the first phase of RiPPLE (2010 and 2011), a DFID supported research partnership. The MoWE and the NWI team.
Figure shows a compilation of two different sets of rural water data.1. MoWE provider data, the officially reported data in the sector used for planning.2. JMP estimates, widely reported globally, based upon household surveys undertaken by the Central Statistical AgencyAround 2000, these different datasets started to diverge significantly, with a difference of over 30 percentage points by 2010. Huge accelerated progress by the sector reflecting the extra investments being made? Or were only the figures accelerating?More recently, JMP results have been significantly revised upwards (to 39% for 2011) while the National WASH Inventory has lead to revision downwards of MoWE results (to 49%). So a 10% difference.
Facilities are in use, can include family wells if adequately protectedStatistical representation at national level: Disaggregation does not extend to local level
Acknowledgements: 2 previous seminars during the first phase of RiPPLE (2010 and 2011), a DFID supported research partnership. The MoWE and the NWI team.