Prepared by Ashley Meek, Engineers Without Borders Canada, Young Samanyika, Ministry of Health, Malawi for the Monitoring Sustainable WASH Service Delivery Symposium, 9 - 11 April 2013, Addis Ababa, Ethiopia.
Unblocking The Main Thread Solving ANRs and Frozen Frames
Development of a National Monitoring and Evaluation Framework for WASH activities in Malawi
1. Development of a National Monitoring and
Evaluation Framework for WASH activities
in Malawi
Monitoring Sustainable WASH Service Delivery
Symposium
April 9-11, 2013
Ashley Meek, Engineers Without Borders Canada
Young Samanyika, Ministry of Health, Malawi
2. Development of a National M&E Framework
Since 2009 there has been a three phase collaboration to
develop a National M&E Framework:
Engineers Without Borders Canada (EWB)
Ministry of Health (MoH),
Ministry of Water Development and Irrigation (MoWDI)
Multiple Districts
Key Development Partners
Main Objectives :
To address the needs of the various stakeholders
To define and harmonize indicators
To develop an easy to use information gathering and
analytical system; and
To capitalise on the existing capacity of the users
4. Context
At the District level,
M&E data collection has not historically been directly
funded by the Ministries.
Data collection has predominantly been an individual
district initiative where indicators are determined as
needed by the district.
Several M&E initiatives and mapping exercises have
been initiated but have not proven sustainable over time.
At the National level
The MoWDI embarked on a Sector Wide Approach
(SWAp) in 2008
Sector Performance Report was initiated in 2010
Data proved difficult to obtain consistently
5. M&E Challenges (stimuli)
Differences in data collection tools.
Non-standard indicator definitions between
departments and organizations.
Non-formalized accountability structures.
Little or no prioritization on data collection as
compared to providing services.
Inadequate knowledge and skills in using some
computer software for data analysis.
Roles and responsibilities not clearly defined.
Systems difficult to maintain and update.
7. Timeline
• EWB began supporting the system initiative through a pilot in partnership with WaterAid
Malawi and three District Water Development Offices
2009 • Basic MS Excel system developed
Phase I
• The systems were replicated with support from EWB to nearly half of the districts in Malawi.
• MoWDI was approved for a grant to develop the SWSME. The project was delayed.
2010
• Malawi National Open Defecation Free (ODF) Task Force (chaired by the MoH) pursues the
development of a comprehensive system for sanitation initiatives.
2011 • EWB, the MoH began a WASH M&E pilot program including Sanitation and Hygiene indicators
Phase II
• MoWDI SWSME project re-initiated
• EWB began consulting with the MoWDI to harmonize with the MoH WASH M&E initiative
2012
Phase III
• MoH pilot complete
• MoWDI pilot including lessons learned from the previous phases.
2013
8. Phase I
In 2009: EWB began supporting the system initiative
through a pilot in partnership with WaterAid Malawi and
three District Water Development Offices
Result : simple Microsoft Excel based system with data
collection by Health Surveillance Assistants (HSAs).
Village level water access was the main unit of analysis.
The system used the pivot table function to perform simple data
analysis
By 2010, the original prototype showed promise and
other districts as well as the MoWDI showed interest.
The systems were replicated with nearly half of the districts in
Malawi.
9. Phase II
reviewed WASH data collection and reporting tools
for various stakeholders and district councils
new reporting tools were developed
data collection processes from village level
developed
analysis templates and mapping functions designed
three learning workshops to review the framework,
indicators and required calculated indicators.
10. Phase III
Collaboration between the MoWDI, MoH and
EWB under the SWSME with several additional
focus areas.
Expansion of capacity building for data collection,
analysis and reporting from the District Environmental
Health Offices to include District and Regional Water
offices.
Strengthening of indicators and creation of analysis
templates for learning institutions, Health Centres and
other public premises.
Establishment of Data Collection Methodologies for
Peri-urban areas.
Explore data verification methods.
Provide data for Sector Performance Report.
12. Step 1. HSA completes community data
form (2 pages) in their villages of
responsibility.
Step 2. Data is entered into a
simple MS-excel spreadsheet.
Step 3. Data automatically populates
an analysis template built using pivot
tables in MS-excel.
Step 4. Data calculated using the templates can be visualized
on a district map outline directly within the MS-excel
program. Each block represents a Traditional Authority
within the District. A map can be generated for each
indicator using a drop down menu.
14. Key Findings of the Framework
1. The monitoring system should be developed
taking existing resource constraints into
consideration to encourage systemic data
collection.
2. Motivation wanes when there is no formal
accountability structure.
3. WASH Data is used by multiple ministries,
district departments and external parties.
4. The system must be simple and easy to use
5. Indicators collected need to be prioritized by
use for service delivery decision making.
15. Key Findings of the Framework
6. Indicators must be clearly defined
7. There is a high probability for error in data.
There needs to be structures in place to verify
authenticity of data.
8. Data collection and particularly data entry is a
large time requirement that needs to be
understood and planned for by management.
9. Resources constraints are a barrier to data
collection activities.
10. Formalized communication structures are
needed between the MoWDI and MoH regarding
M&E
16. Conclusions
The Malawi National M&E framework development has
enabled a system that is adaptable, easy to use, and
relevant to the sector players who use it,
Integration of the various initiatives resulted in a Phased
approach where the lessons learnt from each step can be
built upon instead of being continuously repeated from
scratch.
By building the Framework on existing capacity (including
financial, human resource, and time constraints) of the
users it has be easily transferable and utilized to make
decisions and report on WASH service delivery.
Many steps remain to ensure the system moves from
pilot phases to a National M&E Framework
17. Contact Information
Ashley Meek
Engineers Without Borders Canada
Sanitation Coordinator
ashleymeek@ewb.ca
+265 994307077
Young Samanyika
Ministry of Health, Malawi
Principal Environmental Health Officer
(Ministry’s National Desk Officer for WASH)
y_samanyika@yahoo.com
+265 999557170
Editor's Notes
To address the needs of the various stakeholders involved with WASH service delivery;To define and harmonize indicators that are practical and useful for service delivery decision making at different levels of the system;To develop an easy to use information gathering and analytical system; andTo capitalise on the existing capacity of the users by accounting for financial, human resource, and time constraints of enumerators.
this is a simplified model. It doesn't show ORT going directly to the district, it doesn't show the LDF, and it doesn't show external funders that implement directly without using the government structure at all.
Differences in data collection tools. Non-standard indicator definitions between departments and organizations.Non-formalized accountability structures. Little or no prioritization on data collection as compared to providing services. Inadequate knowledge and skills in using some computer software for data analysis.Roles and responsibilities not clearly defined.Systems difficult to maintain and update.
By 2010, the original prototype showed promise and other districts as well as the MoWDI showed interest. The systems were replicated with nearly half of the districts in Malawi.Some showed very strong dedication to monitoring workOthers failed to collect data, failed to update it, or failed to use it for decision making
there is still work to do before a fully-fledged system exists at scale with accountability mechanisms, everyone using the data, etc.