Successfully reported this slideshow.
Building M&E capacity in
community-based HIV programs in
Tanzania:
From diagnosis to assessing impact
Karen Foreit1
, Dawn...
 Total Population: 45 million
 5.1% of adults age 15-49
are HIV-positive.
Background - Tanzania
 Funded by the United States Agency for
International Development to improve monitoring
and evaluation in population, hea...
The challenge(s)
 Inadequate, poorly coordinated M&E systems
 Insufficient human & organizational capacity
 Poor-qualit...
Community volunteers
 Limited literacy
 Do not understand why they
have to collect so much
information
 Often do not fi...
Data Quality Assessments (DQA)
 Measure accuracy of data reports
 Assess underlying M&E systems
 Originally designed fo...
What we found
 Good results for facility-based programs
 Serious defects in community-based programs
 Few had M&E plans...
What we did
 Re-design DQA to support M&E
capacity-building
 New sampling procedures
 New data collection instrument
 ...
 Stratify sites:
 By size (large vs. small) and
 By location (close to vs. far
away from regional office/HQ)
 Randomly...
 Community Trace and Verify
 Visit “claimed” beneficiaries
 Ask what services they received
 Compare beneficiary answe...
Baseline assessment
 Questions specific to the level being
assessed
 Document every question
 4 point scale for system ...
 Move workshops closer to sites
 Group participants by common
problems
 Follow training with
individualized mentoring
...
Objectively measure change
 “Mini” DQA
 Initial focus on low-scoring areas
 Expand from sites visited at baseline to ot...
Plan for sustainability
Standard Operating Procedures
 Team structure, roles and
responsibilities
 Procedures for implementing
at different leve...
Structured report format
 Text and analysis template
 Standardized color scheme
and lay-out
 External editor
Build local capacity
 Transition to local organization with
backstopping from MEASURE Evaluation
staff
 Report writing c...
Technical Support to Government
 Technical assistance to CHMTs mainly on
enhancing data quality and data use at District
...
Results
 CTV effectively pinpoints strengths and
weaknesses in data collection
 Measurable improvements in M&E plans,
pe...
Also…
 Partners allocate
specific M&E budget
 Hired M&E staff
 Supportive supervision
and capacity building to
lower le...
Conclusions
Routine DQA augmented with CTV is an
effective platform to assess data quality and
program coverage of communi...
MEASURE Evaluation is a MEASURE project funded by the
U.S. Agency for International Development and implemented
by the Car...
Upcoming SlideShare
Loading in …5
×

Building M&E capacity in community-based HIV programs in Tanzania: From diagnosis to assessing impat

363 views

Published on

Presented by Zaddy Kibao at the March 2014 AfrEA Conference.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Building M&E capacity in community-based HIV programs in Tanzania: From diagnosis to assessing impat

  1. 1. Building M&E capacity in community-based HIV programs in Tanzania: From diagnosis to assessing impact Karen Foreit1 , Dawne Walker2 , Mari Hickmann1 , Zaddy Kibao1 1 Futures Group, 2 Effective Development Group Africa Evaluation Association March 2014, Yaounde, Cameroon
  2. 2.  Total Population: 45 million  5.1% of adults age 15-49 are HIV-positive. Background - Tanzania
  3. 3.  Funded by the United States Agency for International Development to improve monitoring and evaluation in population, health and nutrition.  Operational in Tanzania since 2003.  Supports M&E system strengthening and capacity building in health and social welfare programmes. Background – MEASURE Evaluation
  4. 4. The challenge(s)  Inadequate, poorly coordinated M&E systems  Insufficient human & organizational capacity  Poor-quality data  Limited data use for planning & management  Community-based programs staffed with semi-literate volunteers
  5. 5. Community volunteers  Limited literacy  Do not understand why they have to collect so much information  Often do not fill out forms until after the visit
  6. 6. Data Quality Assessments (DQA)  Measure accuracy of data reports  Assess underlying M&E systems  Originally designed for facility-based programs  Sampling proportional to size  “Trace and Verify” of client records
  7. 7. What we found  Good results for facility-based programs  Serious defects in community-based programs  Few had M&E plans in place  Households “claimed” as served reported they had not been visited  Declining data quality with distance to HQ
  8. 8. What we did  Re-design DQA to support M&E capacity-building  New sampling procedures  New data collection instrument  Baseline needs assessment  Tailored training and mentoring  Objectively measure change  Plan for sustainability
  9. 9.  Stratify sites:  By size (large vs. small) and  By location (close to vs. far away from regional office/HQ)  Randomly select from all 4 strata  10 or fewer sites, visit all sites New sampling procedures
  10. 10.  Community Trace and Verify  Visit “claimed” beneficiaries  Ask what services they received  Compare beneficiary answers to activity reports New data collection instrument
  11. 11. Baseline assessment  Questions specific to the level being assessed  Document every question  4 point scale for system assessment for greater precision  Section on data use for all levels
  12. 12.  Move workshops closer to sites  Group participants by common problems  Follow training with individualized mentoring  Advanced workshops for partners with strong basic M&E skills Tailored training and mentoring
  13. 13. Objectively measure change  “Mini” DQA  Initial focus on low-scoring areas  Expand from sites visited at baseline to other sites in the program  Expand to re-assess all areas of the system assessment
  14. 14. Plan for sustainability
  15. 15. Standard Operating Procedures  Team structure, roles and responsibilities  Procedures for implementing at different levels  Debrief templates  GIS guidance  Spot check guidance  Report Writing
  16. 16. Structured report format  Text and analysis template  Standardized color scheme and lay-out  External editor
  17. 17. Build local capacity  Transition to local organization with backstopping from MEASURE Evaluation staff  Report writing course for contractor staff  Greater participation in sampling and fieldwork plan
  18. 18. Technical Support to Government  Technical assistance to CHMTs mainly on enhancing data quality and data use at District level  Strengthening of Community Care National M&E system in Mainland and Zanzibar
  19. 19. Results  CTV effectively pinpoints strengths and weaknesses in data collection  Measurable improvements in M&E plans, performance and data quality  28 IPs assessed, 23 received multiple rounds  10 undertake internal DQA  Slow progress in building local capacity to independently conduct DQA
  20. 20. Also…  Partners allocate specific M&E budget  Hired M&E staff  Supportive supervision and capacity building to lower levels
  21. 21. Conclusions Routine DQA augmented with CTV is an effective platform to assess data quality and program coverage of community programs, guide assistance to strengthen individual and organizational M&E capacity, and measure the impact of training and mentoring for M&E.
  22. 22. MEASURE Evaluation is a MEASURE project funded by the U.S. Agency for International Development and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group International, ICF, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. Government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide.

×