MEASURE EvaluationMEASURE Evaluation works to improve collection, analysis and presentation of data to promote better use of data in planning, policymaking, managing, monitoring and evaluating population, health and nutrition programs.
MEASURE EvaluationMEASURE Evaluation works to improve collection, analysis and presentation of data to promote better use of data in planning, policymaking, managing, monitoring and evaluating population, health and nutrition programs.
3. Troubleshooting
If you lose connectivity
Re-enter the meeting room by clicking on the webinar link
provided
If you have trouble with audio
Make sure your speakers are on. The speaker button at the
top of the screen should be green.
Hover your mouse over the host and choose “private
chat” to send a troubleshooting question
Send an email to glbitar@unc.edu
4. Tips for Participating in the
Discussion
If possible, save questions for the discussion part of
the webinar.
A recording of the webinar will be made available at
http://www.cpc.unc.edu/measure/networks/childstatu
snet
6. What do community programs
provide?
Training and capacity-building
Support to households
Support to individuals
Home-based care
Community awareness-raising
…
7. How are these services provided?
Directly by international Implementing
Partners (IPs)
Through subgrants to civil society
organizations (CS0s) and smaller NGOs
By unpaid community volunteers
8. What are community programs
required to report?
Number of eligible clients who received food and/or
nutrition services
Number of eligible OVC who received a minimum of
one CORE care service
Number of eligible adults and children provided with
home-based care services
Number of vulnerable households with at least one
OVC/MVC or PLHIV provided a minimum of one
economic opportunity/strengthening support
9. MEASURE Evaluation approach to
M&E capacity-building
Three-step process:
1. Assessment
2. Training, mentoring and technical
assistance
3. Re-assessment
10. Community-based
programs are
expected to
produce
comparable data
to facility-based
programs, but
without
comparable
physical
infrastructure and
human resources
11. Multiple levels of data quality
issues
Data collection
Data entry
Data analysis and use
12. Responding to the special needs
of community-based programs
Shortcomings of trace-and-verify
methodology
Developed Community Trace and Verify
(CTV)
13. Community Trace and Verify (CTV)
Covers minimum package of services
Short (10-minute) survey of caretakers
LQAS
sampling
methodology
Pass-fail
scoring
14. 005 Does [Name] have a birth certificate? Yes…1
No…2 → 007
Don’t know …3
006 Could you show us the birth certificate? Yes, birth certificate shown…1
No…2
007 Is the family enrolled with the Community Yes…1
Health Fund? No…2 → 011
Don’t know …3 → 011
008 Does the family have a Community Health Yes…1
Fund card? No…2 → 009
Don’t know …3 → 009
009 Could you show me the community health Yes, card shown…1 → 011
fund card? No…2
010 Could you show me the receipt? Yes, receipt shown…1
No…2
011 Have you heard about the Most Vulnerable Yes…1
Child Committee? No…2
Don’t know …3
012 Has [Name] been visited by a Most Yes…1
vulnerable Child Committee member or No…2 → 014
Volunteer in the past six months? Don’t know …3 → 014
15. CTV in Practice
Piloted with Africare
Introduced to WEI
TA to other OVC IPs
Incorporated into the Department of Social
Welfare MVC M&E Plan
16. Responding to the special needs
of community-based programs
Fostering data demand and use
Developing participatory monitoring and
evaluation (PM&E)
17. Participatory M&E
Low literacy tool
Community identifies priorities
Establishes progress indicators
CBO assists with regular monitoring
18. Establishing priorities
Seed (lowest priority)
Sapling
Tree
Fruiting tree (highest priority)
19. Step 1: Vote
Objectives/ Values
Community Home based Care Service
2 4 3 1
Providers availability when needed
Clients recovery from lost to follow up 4 0 2 4
PLWHA trained on nutritional aspects
3 3 3 1
and food preparation
Availability of pain killer drugs 4 2 3 1
Availability of antifungal drugs 1 0 7
2
PLWHA received economic
2 3 2 3
strengthening support
Stigma and discrimination has been
5 2 3 0
reduced
20. Step 2: Score and rank
Average % Highest priority
Objectives/ Values
Score Rank Score Rank
Community Home based Care Service
2.3 8 10% 8
Providers availability when needed
Clients recovery from lost to follow up 2.8 3 40% 2
PLWHA trained on nutritional aspects
2.2 9 10% 9
and food preparation
Availability of pain killer drugs 2.1 10 10% 10
Availability of antifungal drugs 3.0 2 20% 5
PLWHA received economic
2.6 6 30% 4
strengthening support
Stigma and discrimination has been
1.8 12 0% 12
reduced
21. Participatory M&E
Easy to implement
Creates community
ownership of information
Sets the stage for follow-
up
22. Measurable changes
Increased M&E
budgets
More M&E staff
IPs conducting their
own DQAs
Supportive
supervision and
capacity building
23. Challenges abound!
Pressure to expand services vs. attention to
M&E
Data collection: the weak link in the chain
Overly-ambitious targets
High-level demands for detailed reporting
data
24. Can (should?) community
programs ever report monthly?
Alternatives to the facility model
Simple forms for case management
Semi-annual or annual cluster-sample
surveys to track coverage
Representative population surveys to
measure changes in status
26. MEASURE Evaluation is a MEASURE project funded by the U.S.
Agency for International Development and implemented by the
Carolina Population Center at the University of North Carolina at
Chapel Hill in partnership with Futures Group International, ICF
Macro, John Snow, Inc., Management Sciences for Health, and
Tulane University. Views expressed in this presentation do not
necessarily reflect the views of USAID or the U.S. Government.
MEASURE Evaluation is the USAID Global Health Bureau's
primary vehicle for supporting improvements in monitoring and
evaluation in population, health, and nutrition worldwide.
Visit us online at http://www.cpc.unc.edu/measure
Editor's Notes
Depending on the setting, community programs provide a range of assistance and services.Training for LGAs, CSOs and volunteersEconomic, nutritional and psychosocial support to households made vulnerable by HIV and AIDSIndividual support to eligible children and othersHome-based care Community awareness-raising around HIV prevention, care and support, child protection and other social and legal issues
In the name of sustainability, USAID/Tanzania is pushing for less direct service delivery by international IPs and more local CSO involvement.CSOs typically cover a district and are located in the district capital.Day-to-day services in the communities are typically provided by unpaid community volunteers.
Most of these have to be disaggregated by age and sex
Assessment: Rely primarily on the Data Quality Assessment approachM&E systems, procedures, staff in placeInformation flow and consolidation from service point to HQInformation audit: trace-and-verify, compares services data captured at delivery point with data reported up the chainTraining, mentoring and technical assistanceWorkshops tailored to DQA results, group IPs by service areas and common M&E issues.Individualized mentoring and capacity-building plansGIS, DDU workshops for advanced IPsRe-assessment: mini-DQAWere changes seen in areas selected for strengthening?
Data collection:Primary data collectors are often unpaid volunteersLow literacy levels and aggregation skills of primary data collectorsData entry:Transport issues to get the data from village level to ward/districtStorage/filing issuesLack of time to input data into databases due to workloadLack of computers or computer knowledgeData analysis and use:Lack of understanding as to why the data are collected and how they can be important to them
Spot checks during Data Quality Assessments revealed that OVC were not receiving the services written in the registers (poor data quality for the routine data)There is little systematized supervision/checks of volunteers who delivered services to OVC
Uses low literacy toolsTakes a day to implement the first time Creates ownership of information at community levelSets the stage for progress monitoring for quarterly or semi-annual follow-up