Evaluating Integrated Community Health Programs

Conducting the Evaluation
Anbrasi Edward, PhD
Johns Hopkins University
The material in this video is subject to the copyright of the
owners of the material and is being provided for educational
purposes under
rules of fair use for registered students in this course only. No
additional copies of the copyrighted work may be made or
distributed.
Planning the Evaluation
3
Planning for the Evaluation—1
► Design instruments, pretest, translate,
maps, population data for selection of
clusters, identify trainers and supervisors,
develop guidelines for training, invitation
list for dissemination
► Survey dates: KPC surveys, participatory
evaluation, dissemination meetings

4
Planning for the Evaluation—2
► Select evaluation task team—invitation to
participants; logistics for evaluation;
organize additional data to be reviewed
during evaluation exercise
► Select external consultant to serve as
team leader; obtain donor approval
► Analysis
► Dissemination plan
5
Define the
Purpose of the
Evaluation
Summarize four to five objectives
► If the program met the
stated goals and objectives
► The effectiveness of the
technical approach
(soundness of the program
design and valid indicators
to measure results; use of
data for decision making)

► Capacity-building efforts
within the project and
among partners
(specifically those
enhancing sustainability)
► Lessons learned from the
program (emphasis is on
innovative activities—those
which are transferable and
contribute to sustainability
of achievements)
► A strategy for application
and communication of
these lessons both within
the organization and to
partners
► Provide recommendations
for follow-up or future
program implementation
6
Evaluation Methodology
► Team leader
► Facilitation skills, familiar with context, program (HSS,
RBM, HIV, CHIS, etc.), culture,
language, advocacy or marketing skills, credibility with
scientific community

► Team members (title and role)
► MOH, donor representatives: USAID, UNICEF, community
leaders, other NGOs
► Data collection and analysis; observation checklist; interview
guide
► Site visits (purposeful sampling)
► KPC survey
► Other surveys or questionnaires
► Feedback session and report
7
Selecting Sites
for Evaluation
8
Sample
Evaluation
Schedule
► Day 1: team leader meets with stakeholders to discuss
objectives for program evaluation
► Day 2: project site—debrief project staff, plan evaluation,
logistics, review data and project records
► Day 3–4: design KII/FGD guidelines, translate
► Day 5–6: teams conduct site visits

► Day 7–8: analyze information, make recommendations,
document lessons learned
► Day 9–10: report summary findings and key
recommendations,
local and national stakeholders
9
Evaluation Costs
► Data collection (KPC survey or qualitative studies)
► Field visits to conduct observations, interviews
► Training venue, meals, accommodation for evaluation team,
per diem (if MOH)
► Stakeholder dissemination meetings: AV, accommodation,
meal arrangements
► External evaluator
► Report publication
10
Analyzing Information—Drawing Conclusions
► Are the objectives relevant in the program context?
► Were program objectives and targets achieved?

► If yes, what strategies were employed? At what cost?
► Constraints and successes—promising practices are
identified
► Conclusions and recommendations
► Develop action plan to improve performance
11
Evaluation Report Outline—1
► Summary
► Evaluation methods, achievements, constraints, conclusions,
and recommendations
► Results
► Comparison of baseline and final data, achievements,
constraints
► Community awareness and demand for services
► Service utilization, coverage, quality (provider performance,
essential drug availability,
etc.)
► Effectiveness of BCC strategies, sustainability, scale up
12
Evaluation
Report

Outline—2
► Program management
► Planning; protocols, manuals
► HR: supervision, training, satisfaction, turnover
► Technical support from HQ, MOH, academia, other
► Financial management—variance, cash flow
► Logistics management: supplies, transport
► Information management
► Stakeholder management and capacity building
► Lessons learned, sustainability, phase out plan, community
empowerment
► Conclusions and recommendations
► Appendices: team members, sites, survey reports, data forms
13
Writing and Reviewing SOW
Source: IEG. (2011). Writing terms of reference for an
evaluation: a how-to guide. Retrieved September 10, 2018, from
https://siteresources.worldbank.org/EXTEVACAPDEV/Resourc
es/ecd_writing_TORs.pdf
https://siteresources.worldbank.org/EXTEVACAPDEV/Resourc
es/ecd_writing_TORs.pdf
14
USAID Final

Evaluator
Requirements
► Proven expertise and leadership in integrated community-
oriented reproductive, MNCH projects
► Conduct of evaluations using mixed methods
► Experience with design, collection, and analysis using
applied
research methods
► Familiarity with public health system in Rwanda
► Demonstrated ability to communicate with and lead a team of
stakeholders, staff, and national experts in participatory
evaluation
► Familiarity with USAID programming
► Skill or familiarity with cost analysis methods
► Excellent analytical and writing skills (English)
15
Key Tasks of
Evaluator
► Review project documents and resources
► Refine the evaluation objectives and key questions
► Develop the field evaluation schedule and assessment tools

► Train enumerators and team members on objectives and
process of the evaluation including evaluation tools
► Lead the team to complete the collection, analysis, and
synthesis of information on program performance
► Interpret results and draw conclusions, lessons learned, and
recommendations regarding project outcome
► Lead an in-country debriefing meeting with stakeholders
► Prepare and submit report before September 1
16
USAID Guidelines for Final Evaluation
Source: USAID. (2013). Guidelines for final evaluation.
purposes under
distributed.
Example Projects
2

Kilifi Project—Mid-Term Evaluation
Indicators Baseline KPC LQAS
2004 2005 2006 2007
Immunization - - - -
Proportion of children fully immunized 62 65 55 68
Pneumonia - - - -
Child with cough or difficult breathing accessed care from
HF in 24 hour
87 90 88.5 88
Malaria control - - - -
Child slept under an ITN previous night 21 39 83 76
Women received IPT for previous pregnancy 39 37 61 54
Child receiving correct treatment in 24 hours 18 55 64 NA
3
Project Performance
- Indicators Baseline KPC LQAS
2004 2005 200
6

2007
HIV /
AIDS
Mothers aware of how to prevent HIV 41 NA NA 48
Mothers utilized VCT services 18 NA NA 57
Nutrition Child received vitamin A 61 NA NA 86
Diarrhea
HW before feeding/after defecation 4 39 39 27
Households using chlorine 1 11 16 13
Children with diarrhea received CRS 31 49 48 38
Children received increased fluids 48 64 63 75
4
Quality of Care:
HFA
Case Management Observations Baseline HFA
%
Mid-term
HFA %
Assessment of sick child - -

All danger signs 21% 6%
All major symptoms 26% 33%
All assessment tasks for ARI 26% 43%
All assessment tasks for diarrhea 2% 21%
Correctly assessed for nutrition 5% 9%
All assessment tasks for fever 12% 8%
Temperature was checked before Rx NA 80%
Weighed before clinical observation 71% 90%
Immunization cards checked 53% 61%
Assessed correctly by health worker 41% 27%
5
Sustainability of
C-HIS—1
Mozambique—Community Vital Registration and HIS
6
Sustainability of
C-HIS—2

Mozambique—Community Vital Registration and HIS
7
Pioneering the Care Group Model in Rwanda
8
Rwanda
Strategic
Stakeholder
Engagement—1
9
Rwanda
Strategic
Stakeholder
Engagement—2
10
Rwanda
Integrated CG
Model

11
Supervision of
Community-
Based Health
Interventions
12
Integrating
Community HIS
with District
Statistics
13
World Vision Multi-Country Research Study
► Five-year multi-country multi-site
mixed methods research study on
impact of integrated package of
community-based interventions for
MNCH
► 2012–2017, $3.3 million in
partnership with four country
research institutions
Images: World Vision International. (n.d.). Child health and
nutrition impact study: an overview. Retrieved September 11,
2018,
from

https://www.wvi.org/sites/default/files/CHNIS%202%20page%2
0study%20overview.pdf
https://www.wvi.org/sites/default/files/CHNIS%202%20page%2
0study%20overview.pdf
14
Sustainable Development Goals and CHNIS
15
Disease Burden for IMCI Conditions, Immunization, Nutrition
Outcomes
Reported (past 2 weeks) Cambodia (n=3813) % Guatemala
(n=4096) % Kenya (n=6407) % Zambia (n=2678) %
Diarrhea 23.3 17.8 8.0 16.1
Cough or difficult breathing 19.9 15.7 5.4 9.7
Fever 46.8 22.1 15.1 22.2
Full immunization 56.1 66.5 39.3 63.1
Measles vaccination 62.9 70.8 54.6 70.8
Underweight (<-2SD) 39.9 26.9 14.8 11.1
Severe underweight (<-3SD) 19.6 11.5 5.8 2.6
Stunted (<-2SD) 47.5 48.3 40.3 37.4

Severe stunting (<-3SD) 28.6 20.7 22.8 14.7
Wasted (<-2SD) 6.0 10.6 11.3 3.8
Severe wasting (<-3SD) 1.4 5.1 5.8 1.3
Anemic (<11g/dl) 74.8 - 57.2 -
Severe anemia (<7g/dl) 1.5 - 4.0 -
16
Quality of Screening and Assessment, IMCI
17
Child Caretaker Counseling
18
Stakeholder
Perspectives—1
► KII and FGD
► Mothers and caretakers
► Volunteers
► Heath center staff
► Hospital staff
► Community development council
► TBA, midwives, other community providers

► Pastors, teachers, drugs sellers, shop keepers
► Representatives of clubs: women, youth, anti-SIDA
► Program staff
► District director
► Other partners and NGOs
19
Stakeholder
Perspectives—2
20
Kilifi, Kenya—
Formative
Assessments
Community stakeholders FGD/KII (n) Participants (n)
Mothers, caretakers 11 107
Fathers 10 92
Care groups 12 114
CHW and CHW TOT 12 83
VHC 11 70
Head teacher 5 5
Village chief 6 6

CTC 3 31
DHMT 1 4
Health workers and in charge 4 4
PHO/PHT 4 4
DHC 5 35
AKS/PSI/Ministry of Agriculture 3 3
21
Kilifi, Kenya—Mothers, Caretakers
► Fathers play a minimal role in child care, not interested in
clinic visits
► Very few access village bank services, not aware of puppet
shows
► Financial constraints in purchasing food supplies for PD
hearth
► Distance to HF is a major barrier to service utilization
► “Before the project started, we were in the ‘dark’ regarding
health issues but now we are
in the ‘light’! We no longer go to consult the traditional
healers.” –Judith, Bundacho
Village

22
Rwanda Fathers
FGD
► Majority mentioned benefits of nutrition weeks and balanced
food
► Most utilize health services as a result of CHW and CG
advocacy
and follow medical advice and prescriptions
► Claimed diseases decreased since the initiation of the
program
► Many appreciated the strategies to understand health issues
and even resolve disagreements at home
► Interesting accounts of improved relationships with their
wives due to:
► Improved hygiene, reduced complaints in the home
► Provision of health insurance, give women security and
lessened pressure on men, hence less conflict in the home
► FP lessons have decreased the burden of raising a large
family they cannot
23
Kilifi, Kenya—Village Health Committee

► Equal representation of both male and females in VHC
► Willing to sustain project activities even after phase out
► Provide bicycles, badges, uniforms and stationery
► More outreach visits for remote areas
► “Before the project many children died of diarrhea and we
did
not know the cause. After we were trained we realized it was
caused by drinking dirty water. The project provided water
guard
and Pur which we now use. We can now see that diarrhea has
reduced. We are also trying to build toilets as a result of the
project.” –Fatumah Ngao, Mlango Village
24
Kilifi, Kenya—Head Teacher
► Most schools have regular health
activities
► Reduced illness, improved hygiene, less
absenteeism among school children
► Teachers mobilize community for health
promotion meetings
25

Kilifi, Kenya—Health Providers and In Charge
“The increased participation of the CHWs
and trainers has led to a reduction in the
work load at the health facility” Wahome,
Clinical officer in charge Vitengeni Health
Centre
“The transport provided by the project
during the health outreaches has enabled us
to surpass our set immunization targets”
Nzioka, PHO
26
Kilifi, Kenya—DHMT
► “Before the project started we had frequent stock outs of Ab,
now after the EPI logistic
management support we no longer have stock outs in the
dispensaries.”
► “Issues like the ‘talking nets’ can be effectively addressed by
opinion leaders like the
CORPS who can change the perception of the community.”
► “The 10 households/homesteads model is an excellent
strategy to sustain health
promotion activities which is now included in the MOH policy.
Want to expand this
concept to other districts.”

27
Rwanda—Community Voices
► “After attending NW, our husbands are now helping us
getting ingredients for household
food preparation and even in preparing them. Their attendance
has even helped to
improve our relationships at home.” A Mother in NW, Mugano
Sector
► “Before I knew that a pregnant woman eats once a day. But
today, I know that she has to
eat three times per day. I was also not used to drinking clean
water but I now drink boiled
water.” Clementine, 22, Mushubi Sector
► “I have learnt how to breastfeed my baby. Before, I would
wait for a long time and
breastfeed when I had time, or when she cried, I would
breastfeed her while doing other
things. I now enjoy breastfeeding often and my baby enjoy it as
well.” Said
enthusiastically by Jacqueline, 24, from Mushubi Sector
28
Rwanda: Nyamagabe Vice Mayor and Health Director—1
► CG model’s significant contribution in district’s goal
to eradicate malnutrition and stunting (was at 53%
but has reduced to 41% at last recording)
► Project has helped the district to reduce infant and

child mortality
► Women appreciate iCCM and there is a great level
of acceptability
► Even when they are referred to a health facility,
they go reluctantly, as they prefer to be treated by
CHWs in their neighborhood
29
Rwanda: Nyamagabe Vice Mayor and Health Director—2
► Significant progress has been observed by the district
in key indicators such as ANC, facility delivery,
indicators on nutrition status of children under 2, etc.
► Improved community hygiene—people now
understand the value of using and maintaining a
clean pit latrine
► The CG infrastructure was used for advocacy against
GBV, 100 percent coverage, reached all households
30
Afghanistan—
RBF Evaluation
31

Mozambique
Final Evaluation
Performance indicator Baseline Final
Full immunization 74% 91%
U5 who slept under an ITN previous night 0.3% 85%
Rx for fever < 24h in HC/HF 28% 90%
Child treated with ORT for diarrhea 53% 94%
Child received extra food for 2w following
diarrhea
19% 87%
Rx for fast/difficult breathing < 24h in
HC/HF
2% 87%
Severe MN 13% 7.2%
Trained providers in district MOH < 1% 100%
Delivery by SBA 66% 88%
Latrine coverage 28% 75%
32

Mortality
Research
Studies
Assessments
- All household census data (HIS for 2002–2003)
- 3/2000-2/2001 3/2001-2/2002 3/2002-2/2003 3/2000-2/2003
Total births 4557 6244 6410 17211
Total child
deaths
543 408 286 1237
Total infant
deaths
318 231 153 702
Child death
rate
119 65 45 71.9
Infant death
rate
70 37 23.9 40.8
33
Comparison of Mortality Estimates

Source: Edward, A., Ernst, P., Taylor, C., Becker, S., Mazive,
E., & Perry, H. (2007). Examining the evidence of under-five
mortality
reduction in a community-based programme in Gaza,
Mozambique. Trans R Soc Trop Med Hyg, 101(8), 814–822.
Retrieved from
https://doi.org/10.1016/j.trstmh.2007.02.025
https://doi.org/10.1016/j.trstmh.2007.02.025
34
Cambodia—Full
Immunization
(12–23m)
35
Cambodia Final
Evaluation
Use of Iodized
Salt
36
Cambodia—Verbal Autopsy 0–59m
37

Final Evaluation Promising Practices
► Cross cutting: “Traditional healers or drug vendors interested
in participating in health
promotion”
► Enhancing motivation: “Community micro-projects and
incentive mechanisms enhance
sustainability and ownership of project activities and outcomes”
► Community training: “Regular supervision of CHWs was
instrumental in CHW motivation”
► Organizational learning: “Collaborations with AKF and
Malaria Research Institute
strengthened district health management and prevention of
malaria”
► CG infrastructure; integration for emergency preparedness
strategies
38
Theory of Change Concepts
► Process of desired change by making explicit the way we
think about a current situation or
problem, its underlying causes, the long-term change we seek,
and what needs to happen
in order for that change to come about
► Product that contains a set of hypotheses, outcomes,
assumptions, and indicators that

make up causal pathways of change needed to bring about a
desired long-term goal
► Causal loop analysis
Adapted from Food Security and Nutrition Network. (2017).
Theory of change training curriculum. Retrieved September 12,
2018,
from https://www.fsnnetwork.org/theory-change-training-
curriculum
https://www.fsnnetwork.org/theory-change-training-curriculum
39
Theory of
Change
Source: Mobile Alliance for Maternal Action (MAMA). (2012).
Global monitoring and
evaluation network [PDF]. Retrieved September 12, 2018.
https://www.mhealthknowledge.org/sites/default/files/MAMA_
Global_MEPlan_FINAL_all_0.pdf
purposes under
distributed.
Personal Reflections

2
Evidence of Local Innovations…
Source: Litsios, S. (2002). The long and difficult road to Alma-
Ata: a personal reflection. International Journal of Health
Services,
32(4), 709–732. Retrieved from https://doi.org/10.2190/rp8c-
l5ub-4raf-nrh2
https://doi.org/10.2190/rp8c-l5ub-4raf-nrh2
3
Read…the Fine
Print
4
Read…the Fine
Print—1
5
Read…the Fine
Print—2

6
Addressing Food Security and AIDS Prevention through Youth
Clubs
7
Youth Club Evaluation: CDD and Malaria
8
Knowledge on
MN and
HIV/AIDS
9
Disseminating Results
► Multi-stakeholder
► Content and delivery designed for strategic audience
► Donor (UNICEF, USAID, taskforce, other)
► MOH: district, national
► PVO staff, other NGOs
► VHC, VDC, CHW, etc.
► Community

10
Stakeholder
Dissemination
11
Policy Makers
► Project HQ offices
► Results oriented: lessons learned and best practices
► Problems to be addressed
► Health system
► Coordination of local health activities
► Give credit to their contributions
► National—presentation if results have major implications
12
Kibogora
District Hospital
Director,
Rwanda
13
Kibogora
District Hospital

14
Scale Up—Care Group Model
► Dissemination of results: local, conferences
► Research publication—role of academic stakeholders
► Advocacy with donors for extension, scale up, and
integration into national policy
► CG Model
► Mozambique: national scale up (USAID/UNICEF)
► Cambodia: USAID Mission, CS taskforce
► Rwanda: scale up, selected Provinces
► USAID: Coregroup consortium > 40 NGO’s
► CHIS; Kenya: national scale up
► CSC; Afghanistan, national scale up
15
Monitoring and Evaluation’s Raison d'Être—Grun
Source: Grun, R. E. (2006). Monitoring and evaluating projects:
a step-by-step primer on monitoring, benchmarking, and impact
evaluation. World Bank, Washington, DC. Retrieved September
12, 2018, from
https://openknowledge.worldbank.org/handle/10986/13640
https://openknowledge.worldbank.org/handle/10986/13640
16

Performance
Metrics
“You cannot manage what you cannot measure” –Attributed to
Lord Kelvin
“Not everything that can be counted counts, and not everything
that
counts can be counted” –Attributed to Albert Einstein
17
Evidence from PHC—Informing Policy
“I am like this lamp, lighting the lamp of
better health. Workers like me can light
another and another and thus encircle the
whole earth. This is ‘Health for All’”
–Muktabai Pol, village health worker
Jamkhed, India, GHC conference, DC
27117Conducting the EvaluationPlanning the
EvaluationPlanning for the Evaluation—1Planning for the
Evaluation—2Define the Purpose of the EvaluationEvaluation
MethodologySelecting Sites for EvaluationSample Evaluation
ScheduleEvaluation CostsAnalyzing Information—Drawing
ConclusionsEvaluation Report Outline—1Evaluation Report
Outline—2Writing and Reviewing SOWUSAID Final Evaluator
RequirementsKey Tasks of EvaluatorUSAID Guidelines for
Final Evaluation27118Example ProjectsKilifi Project—Mid-
Term EvaluationProject PerformanceQuality of Care:
HFASustainability of C-HIS—1Sustainability of C-HIS—
2Pioneering the Care Group Model in RwandaRwanda Strategic
Stakeholder Engagement—1Rwanda Strategic Stakeholder

Engagement—2Rwanda Integrated CG ModelSupervision of
Community-Based Health InterventionsIntegrating Community
HIS with District StatisticsWorld Vision Multi-Country
Research StudySustainable Development Goals and
CHNISDisease Burden for IMCI Conditions, Immunization,
Nutrition OutcomesQuality of Screening and Assessment,
IMCIChild Caretaker CounselingStakeholder Perspectives—
1Stakeholder Perspectives—2Kilifi, Kenya—Formative
AssessmentsKilifi, Kenya—Mothers, CaretakersRwanda Fathers
FGDKilifi, Kenya—Village Health Committee Kilifi, Kenya—
Head TeacherKilifi, Kenya—Health Providers and In
ChargeKilifi, Kenya—DHMTRwanda—Community
VoicesRwanda: Nyamagabe Vice Mayor and Health Director—
1Rwanda: Nyamagabe Vice Mayor and Health Director—
2Afghanistan—RBF EvaluationMozambique Final
EvaluationMortality Research Studies AssessmentsComparison
of Mortality EstimatesCambodia—Full Immunization (12–
23m)Cambodia Final Evaluation �Use of Iodized
SaltCambodia—Verbal Autopsy 0–59mFinal Evaluation
Promising PracticesTheory of Change Concepts Theory of
Change27119Personal ReflectionsEvidence of Local
Innovations…Read…the Fine PrintRead…the Fine Print—
1Read…the Fine Print—2Addressing Food Security and AIDS
Prevention through Youth ClubsYouth Club Evaluation: CDD
and MalariaKnowledge on MN and HIV/AIDSDisseminating
ResultsStakeholder DisseminationPolicy MakersKibogora
District Hospital Director, Rwanda Kibogora District
HospitalScale Up—Care Group ModelMonitoring and
Evaluation’s Raison d'Être—GrunPerformance MetricsEvidence
from PHC—Informing Policy
Elements of Project Evaluation
Anbrasi Edward, PhD

Johns Hopkins University
purposes under
distributed.
What Are the Elements of Project
Evaluation?
3
Power of
Measuring
Results
ure results, you cannot tell success from
failure
failure
lure, you cannot correct it

Source: Osborne, D., & Gaebler, T. (1992). Reinventing
government. Penguin Press: New York, NY.
4
Purpose of
M&E Systems
► Track progress, results, and effectiveness of programs
► Provide government officials, development managers, and
civil
society evidence from the investments, improving service
delivery, planning and allocating resources, and informing
management decision-making
► Accountability of resources
► Systems reengineering
► Reward successes and recognize improved performance
► Organizational learning and growth, opportunity for future
funding
5
USAID CLA ► Collaborating, learning, adapting
► Facilitating collaboration internally and with external
stakeholders
► Integrate new learning, innovations, and performance

information into the current strategy to inform funding
allocations, program design, and project management
► Translate new learning and information about changing
conditions into iterative strategic and programmatic
adjustments
► Catalyze collaborative learning, systemic analysis, and
problem solving among communities and institutions to
develop and implement programs that are more effective at
achieving results
6
World Bank Monitoring and Evaluation (M&E) Are
Synergistic—1
► Monitoring
► “A continuing function that uses systematic collection of
data on specified indicators to
provide management and the main stakeholders of an ongoing
development
intervention with indications of the extent of progress and
achievement of objectives
and progress in the use of allocated funds”
► Embodies the regular tracking of inputs, activities, outputs,
outcomes, and impacts
Source: World Bank. (n.d.).

7
World Bank Monitoring and Evaluation (M&E) Are
Synergistic—2
► Evaluation (more detailed, time consuming, resource
intensive)
► Process of determining the worth or significance of a project
to determine the
relevance of objectives, the efficacy of design and
implementation, the efficiency of
resource use, and the sustainability of results
● An evaluation should enable the incorporation of lessons
learned into the decision-
making process of both partner and donor
Source: World Bank. (n.d.).
8
Defining Project Evaluation
► “The systematic collection of information about the
activities, characteristics, and
outcomes of programs, for use by people to reduce
uncertainties, improve effectiveness,
and make decisions” (Patton, 2008)
► “Systematic collection and analysis of information about the
characteristics and outcomes
of strategies, projects, and activities as a basis for judgments to

improve effectiveness,
and/or to inform decisions about current and future
programming” (USAID)
“If you can’t measure it, you can’t manage it” –W. Edwards
Deming
“If you can’t measure it, you can’t improve it” –Peter Drucker
9
Rationale for
M&E
► Well-designed and high-quality evaluation is important to
track
progress, results, and effectiveness of programs
► Appreciate why programs are succeeding or failing, and can
provide insight to adapt and improve performance
► M&E contribute data-based evidence to facilitate strategic
planning, project design, allow mid-term project remedies and
resource decisions, and provide a forum for learning and
growth
► Improve the quality of evaluation metrics, determine
appropriate study designs, answer research questions, and
accommodate external third party evaluations for objective
inference on project performance
10

Evaluation
Elements
► Systemic collection and analysis of data to assess the
conceptualization, design, implementation, and utility of
programs
► Requires data collection at multiple points in time (baseline
and end of project) in order to demonstrate change
► Longer time frame, 3–5 years
► By examining longer-term results, identifying how and why
activities succeeded, failed, or changed, evaluation informs the
design of future projects
► Outcome or impact evaluations, which assess program
achievements and effects
► Evaluation data may include some routine monitoring data
but
require a higher degree of rigor to collect and are gathered by
an external evaluator to the implementing partnership
Source: Rossi, P. H., and Freeman, H. E. (1993). Evaluation: a
systematic approach (5th ed.).
Newbury Park, CA: Sage Publications, Inc.
11
“Drive out
Fear” Edward
Deming

► FEAR, ANTICIPATION, EXCITEMENT
► Evaluation outcomes
► End of project, second funding cycle, cost extension
► Promotion, performance incentives
► Nature of evaluation
► Internal
● Participatory—team leader engages program stakeholders
in all aspects
● Opportunity for problem solving, less threatening
● Bias, validity, generalization
► External, third party
● Academic institutions, independent contractors
● Perform assessments and present to stakeholders
► Accreditation
purposes under
distributed.
Evaluation Principles
2

Evaluation Principles: USAID—1
► Integrated into the design of strategies, projects, and
activities
► Planning for and identifying key evaluation questions during
the planning stage will
improve the quality of strategy development and project design
and guide data
collection during implementation
► Unbiased in measurement and reporting
► Not subject to the perception or reality of biased
measurement or reporting due to
conflicts of interest or other factors
► Relevant
► Address the most important and relevant questions about
strategies, projects, or
activities
3
Evaluation Principles: USAID—2
► Based on evidence-based methods that generate the highest
quality and most credible
evidence that corresponds to the questions being asked, taking
into consideration time,
budget, and other practical considerations

► Foster local capacity
► Consistent with aims of local ownership through strategic
engagement of all
stakeholders, including community participants
► Transparent
► Cascade results downstream with a commitment to full and
active disclosure of results
4
Evaluation Considerations—1
► Program context, objectives, intervention strategies
► External versus participatory, quantitative or formative
► Process or outcome measures
► Data sources: clinical observations, record review,
participant interviews, formative
assessment, focus groups and key informant interviews
5
Evaluation Considerations—2
► Enumeration, sample size, design effect, power calculations
► Sampling: simple random, systematic, stratified, cluster
(multistage), non-probability,

convenience, LQAS
► Clinical, managerial, cost effectiveness measures
► Other considerations
► Primary sampling unit, unit of analysis
► Informed consent procedures, ethical considerations
► Resources: time, personnel, finance
6
Hierarchy of Study Designs
Study design Definition
Randomized control trial
(RCT)
Involves random allocation of the intervention and comparison
(e.g., usual care) to different study groups, including
measurement of the outcome before and after the intervention
has been made.
Non-RCT Observations are made before and after the
implementation of an intervention, both in a group that receives
the
intervention and in a control group that does not.
Case-control Compares people with a specific outcome of
interest (“cases”) with people from the same source population
but
without that outcome (“controls”) to examine the association
between the outcome and prior exposure (e.g.,
having an intervention).

Cohort Participants who do not have the outcome of interest are
allocated (but not randomly) to receive alternative
treatment, therapy, or interventions (experimental group) or the
standard of care or conventional treatment
(comparison group) or no intervention (control group) and then
followed up to determine whether they experience
the outcome of interest.
Cohort One group (pre- and post-): single selected group under
observation, with a careful measurement being done
before applying the experimental treatment and then measuring
after. Two groups (pre- and post-): group of
people with a common set of characteristics or a set of
characteristics that are followed up for a period of time to
determine the incidence of an outcome.
Interrupted time-series Uses observations at multiple time
points before and after an intervention (the “interruption”) in an
effort to detect
whether the intervention has had an effect significantly greater
than any underlying trend over time.
7
Study Designs Continued
Study design Definition
Case or time series Observations are made on a series of
individuals, usually all receiving the same intervention, before
and after an intervention but with no
control group.

Cross-sectional Both exposures and outcomes are measured at a
single point in time, and the prevalence of the outcome is
compared among those with and
without exposure. BEFORE/AFTER MEASURES
Quantitative, other Systematic empirical investigation of social
phenomena via statistical, mathematical, or numerical data or
computational techniques other than
those listed above.
Cost-effectiveness Cost effectiveness analysis (CEA) is a type
of economic evaluation that examines both the costs and health
outcomes of alternative intervention
strategies.
Mixed methods Combines data collection approaches,
sometimes both qualitative and quantitative, into the study
methodology. Some studies combine study
designs, whereas others may have a single overarching research
design but use mixed methods for data collection.
Implementation
science
Scientific study of methods to promote the systematic uptake of
research findings and other evidence-based practices into
routine practice and
to improve the quality (effectiveness, reliability, safety,
appropriateness, equity, efficiency) of care. It includes the
study of factors impacting
behavior at the individual (i.e., provider, patient) and
organizational levels.
Case study The study of selected current phenomenon over time
to provide an in-depth description of essential dimensions and
processes of the

phenomenon. Case studies may involve both qualitative and
quantitative data and may focus on single or multiple cases.
Participatory action
research (PAR)
Systematic inquiry, with the collaboration of those affected by
the issue being studied, for purposes of education and taking
action or effecting
change.
Qualitative Used to generate an understanding of complex,
unquantifiable phenomena, such as people’s experiences or
perceptions.
8
Hierarchy of Evaluation Methods?
► Robust experimental designs to weaker qualitative
techniques?
► Contextual application; no right and wrong approach
► Appropriateness of the strategy, knowledge base, resource
availability, time,
environment, level of analysis
9
Sample Size and Power Determinations

► An important consideration for any evaluation study is
sample size
► Power analysis should be performed to estimate the sample
size needed to detect
program effects with a reasonable level of certainty
► The power analysis can take into account various kinds of
outcome indicators (such as
continuous, binary, and count measures) and the consequent
statistical requirements for
rejecting the null hypothesis assuming small, moderate, or large
effect sizes, as
appropriate
► Standard power calculations can be done using widely
available software
► The longitudinal nature of the measures proposed may
require large samples to
accommodate more independent and dependent variables and to
account for anticipated
attrition from the study
► In impact evaluations, timing is essential—many outcomes of
interest may take months to
demonstrate change (maternal mortality, illness incidence, etc.)
10
Crafting Evaluation Frameworks—1
► Performance indicators
► The logical framework approach

► Formal surveys
► Rapid appraisal methods
► Participatory methods
► Public expenditure tracking surveys
► Cost-benefit and cost-effectiveness analysis
► Impact evaluation
► Not a comprehensive list! Some are complimentary
► Broad and narrow applicability
► Depends on the need of users; the speed for
information and cost
11
► Formal surveys
12

► Formal surveys
13
Performance Indicators
► Measures of inputs, processes, outputs, outcomes, and
impacts for development projects,
programs, or strategies
► Appropriate study design with formal surveys, analysis and
reporting, indicators enable
managers to track progress, demonstrate results, and take
corrective action to improve
service delivery
► Setting performance targets and assessing progress toward
achieving them
► Provides an early warning system to allow corrective action

to be taken
► Effective means to measure progress toward objectives,
facilitates benchmarking
comparisons between different organizational units, districts,
and over time
► Tendency to include too many indicators, or those without
adequate data sources
resulting in expensive, impractical, and underutilized data
14
Configuring Performance Frameworks
► Country demand driven and country led
► Build on existing in-country processes and experience
► Audience: inclusive, involves civil society and other
stakeholders
► Align with leading global innovations
► System framework with a value chain and validated
indicators (avoid collinearity)
► Address epidemiological transition
► Measurement within the scope of the healthcare organization
► Maturity and sophistication
► Mandate, skills, resources and independence to succeed
► Measures, methods, frequency, sampling, weighting
► Financing and cost structure

15
Common
Project Errors
Source: The Project Cartoon. (n.d.). [Cartoon]. CC BY 3.0.
Retrieved September 25, 2018.
http://www.projectcartoon.com/
https://creativecommons.org/licenses/by/3.0/
purposes under
distributed.
Evaluation Components
2
Evaluation Components—1
► Log frame with monitoring and evaluation
(M&E) plan (goal, objectives, indicators,
means of verification, activities,
assumptions)
► Baseline, midterm, final, post-project

(qualitative and quantitative)
► Performance monitoring (HMIS,
supervision)
► Annual self-evaluations—informal,
opportunity to redirect activity plan and
strategy as well as build consensus
► Interim and formative evaluation,
midterm review
► Special studies or evaluations
3
Evaluation Components—2
► Health facility assessments: utilization, quality of care,
capacity, referral
► Household surveys: perceptions, knowledge, care-seeking
behavior
► Community capacity assessment: health councils, traditional
healers, religious leaders
► Organizational learning: cost/unit output, cost effectiveness
analysis
► Stakeholder capacity building: partners, drug sellers
► Research: verbal autopsy, operations research: new products
or drugs, delivery systems
(CHW blood pressure monitoring)
► Process and impact evaluation: public health significance
► Equity, gender, country ownership, “smart” integration, etc.
► Scale: can the project be scaled up geographically to other

regions?
► Sustainability: does the project have strong partnerships and
financing for sustainability?
4
Facility
Performance
Measures
Method Description
Service provision assessment, DHS
measure
National sample, 3–5 years
Rapid—SPA, CSTS Provincial/district, 3–5 years
Service availability mapping, WHO Mapping of service and
resource
availability, national and district level
decision making, annual
Health facility census, JICA Health assets mapping, national, 5
years
FASQ, measure Facility audit of service quality
QIQ, measure Quick investigation of quality
LQAS, (EPI, HIV/AIDS) Lot quality assurance sampling
IMCI—MCE, WHO, Rapid HFA Focus on child health

GFATM, RBM, EPI, RBF, etc. Global initiatives
Modules include system readiness, QoC, provider and client
interviews, donor assistance,
social marketing, record reviews, costing, etc.
Technical areas: MNCH, STI, TB, HIV/AIDS, FP, etc.
Management, financing, motivation, design, sampling, duration
and types of data collection
methods vary
5
Community and
Household
Surveys
Method –
UNICEF MICS Multi-indicator cluster surveys
USAID, DHS Demographic health survey
KPC 30 cluster surveys Knowledge practice coverage
KAP 30 cluster surveys Knowledge attitude practice
PEPFAR, PMI President’s initiatives for AIDS, malaria
WHO verbal autopsy Verbal autopsy for non-clinicians
CS-KPC rapid catch Child survival key performance indicators
PETS, WB Public expenditure tracking surveys

LSMS Living standards measurement surveys
Indonesia FLS Indonesia family life surveys
HEIDE Household expenditure and income data for transitional
economies
IHSN International household survey network
DDP WB development data platform
6
Multidimensional Evaluation Measures—1
► Health system and district team
► Capacity building: training—technical,
managerial, information systems
► Utilization patterns, quality of care,
equity (OOP), efficiency
► Supply chain management; HR,
essential commodities, facility
preparedness
► Community linkages; CHW training and
oversight
► Cost recovery
► Scope, cost, data propriety

► PVO/NGO
► Capacity building; training, field visits,
etc.
► Communication, annual and quarterly
reports
► Staff competency, performance,
satisfaction; oversight, personnel
development, compensation systems,
etc.
► Financial management
► Organizational learning and leadership
7
Multidimensional Evaluation Measures—2
► Program effectiveness
► Achievement of goals and objectives
● Quantitative: MICS/KPC
● Qualitative: KII, FGD, etc.
► Community capacity and health
system capacity
► Sustainability strategy, scale up
► Community
► Participation, empowerment
► Functional community structures:

CHW, VHC, health facility councils and
other interest groups, etc.
8
Global
Reference List
► Guide monitoring of health results nationally and globally
► Reduce excessive and duplicative reporting requirements
► Enhance efficiency of data collection investments in
countries
► Enhance availability and quality of data on results
► Improve transparency and accountability
► Disaggregations that include equity stratifiers (e.g., age and
sex,
geography, socioeconomic status, place of residence)
► Data sources: civil registration, vital statistics systems,
population-based health surveys, facility-generated data,
routine facility information systems, health facility assessments
and surveys, administrative data sources such as financial and
human resources information systems
Source: WHO. (2018). Global reference list of 100 core health
indicators (plus health-related
SDGs), 2018. Retrieved September 26, 2018, from
http://www.who.int/healthinfo/indicators/2018/en/

9
WHO 100 Core
Health
Indicators
Source: WHO. (2018). Global reference list of 100 core health
indicators (plus health-related
SDGs), 2018. Retrieved September 26, 2018, from
10
Impact
Evaluation
► Systematic identification of the effects; positive/negative,
intended/not, on individual households, institutions, and the
environment caused by a program or project
► Helps determine equity, effectiveness, coverage, and
outcomes
► Range from large scale sample surveys and control groups
are
compared before and after, to small-scale rapid assessment
► Measuring outcomes and impacts of an activity and
distinguishing these from the influence of other, external

factors
► Accountability of investments, informing decisions on
whether
to expand, modify or eliminate projects, programs or policies
► Drawing lessons for improving the design and management
of
future activities
► Comparing the effectiveness of alternative interventions
► Expensive, time consuming
11
Cost-Benefit and Cost-Effectiveness Analysis
► Cost-benefit analysis measures both inputs and outputs in
monetary terms
► Cost-effectiveness analysis estimates inputs in monetary
terms and outcomes in non-
monetary quantitative terms
► Informing decisions about the most efficient allocation of
resources
► Identifying projects that offer the highest rate of return on
investment
12

Rapid Appraisal ► Rapid low-cost ways to obtain feedback of
participants and
other key stakeholders
► Provides real time information for management decision-
making, especially at the project or program level
► Provides qualitative understanding of complex
socioeconomic
changes, highly interactive social situations, or people’s values,
motivations, and reactions
► Providing context and interpretation for quantitative data
collected by more formal methods
► Findings usually relate to specific communities or
localities—
thus difficult to generalize from findings
13
Rapid Appraisal
Methods—1
► Key informant interviews: a series of open-ended questions
posed to individuals selected for their knowledge and
experience in a topic of interest
► Interviews are qualitative, in-depth, and semi-structured
and use guides with a list of topics or questions
► Focus group discussions: a facilitated discussion among 8–12
carefully selected participants with similar backgrounds
► Participants might be in the study, program staff, or

stakeholders
► The facilitator uses a discussion guide, note-takers record
comments and observations
14
Rapid Appraisal
Methods—2
► Direct Observation: use of a detailed observation form to
record what is seen and heard at a program site
► Mini-survey: structured questionnaire with a limited number
of
closed-ended questions that is administered to 50–75 people
► Selection of respondents may be random or “purposive”
15
Participatory Evaluation
► Collaborative problem solving
► Contributes to learning and leads to strengthened program
design and implementation
► External expert plays a facilitating role with stakeholders
► Share ideas and encourage team to consider options

16
Participatory
Methods
► Learning about local conditions and local people’s
perspectives
and priorities to design more responsive and sustainable
interventions
► Identifying problems and troubleshooting problems during
implementation
► Evaluating a project, program, or policy—mixed methods
studies
► Providing knowledge and skills to empower poor people
► Stakeholder analysis: develop an understanding of the power
relationships, influence, and interests of the various people
involved in an activity and determine who should participate
and when
17
Constraints of Participatory Evaluation
► Considered less objective because of stakeholder
involvement
► Difficulty in addressing highly technical aspects

► Time and resources for conduct and consensus of an array of
stakeholders
► Risk of domination and misuse by some stakeholders to
further their own interests
However, the benefits of fully engaging stakeholders throughout
the evaluation outweigh
these concerns (Fetterman et al., 1996)
18
Mixed Methods Evaluation
► In addition to determining impact, it is equally important to
understand the
implementation process of the program, the successes and
challenges to bring about
improvements in quality of the program, and learning for
program improvement and
expansion
► Therefore, it is critical that the impact evaluation and
formative research are
systematically aligned
19
Data Quality,
Storage Security
► Validity—data clearly and adequately represent the intended

result
► Integrity—data have safeguards to minimize the risk of
transcription error or data manipulation
► Precision—data have sufficient level of detail to permit
management decision making
► Reliability—data reflect consistent collection processes and
analysis methods over time
► Timeliness—data are available at a useful frequency, are
current, and timely enough to influence management decision
making
Proper data storage and security are critical to protecting data
integrity, optimizing data usability, and safeguarding
potentially
sensitive or personally identifiable information.
20
Data Analysis—1
► Data may require some validation and cleaning before
analysis to ensure they are
accurate and make sense
► Data may require substantial analysis before it is included in
a report
► Type of analysis depends on the kind of data collected and
intended use

21
Data Analysis—
2
► Qualitative data will undergo content or pattern analyses to
see
trends
► Quantitative data may be simple analyses to generate sums or
averages, disaggregated, or they require complex approaches
such as regression analyses or multilevel modeling (logistic,
Poisson, etc.)
► Uses dependent variables designed to reflect the appropriate
level of measurement for each outcome of interest—binary
events, such as birth in a clinic, or other events, such as
number of antenatal clinic visits and number of birth
complications, may be measured with count variables
► Include data analysis plan, software, etc.
22
Presentation and Use of Data
► Data can presented using a variety of visuals: tables, graphs,
and charts as appropriate
► Key findings will be summarized in PowerPoint
presentations

► The data can presented at the monthly and quarterly meetings
within the project;
annually with partners; and end of project with donors and key
stakeholders
► The primary use of the data will be to inform the program
management if changes are
required for program activities
23
Road Map ► Formulate research and program questions, theory
of change
► Configure evaluation framework, determine indicators, data
sources
► Study design, sampling, data collection strategies
► Conduct and manage the evaluation
► Data management
► Results sharing and reporting—use, learning, action plans
24
References ► Monitoring Toolkit (USAID)
► Performance Indicator Reference Sheet (USAID)
► Ten Steps to a Results-Based Monitoring and Evaluation
System

(World Bank)
► Monitoring and Evaluation: Some tools, methods and
approaches (World Bank)
► The Monitoring and Evaluation Handbook For Business
Environment Reform (World Bank)
► Impact Evaluation Toolkit (World Bank)
► Global Reference List of 100 Core Indicators (WHO)
► MAMA Global Monitoring and Evaluation Framework
(USAID)
References retrieved September 26, 2018.
https://usaidlearninglab.org/evaluation-toolkit?tab=1
https://usaidlearninglab.org/sites/default/files/resource/files/cle
ared_-_mt_-_f_and_usaid_ref_sheet_cross_walk.pdf
https://www.oecd.org/dac/peer-
reviews/World%20bank%202004%2010_Steps_to_a_Results_Ba
sed_ME_System.pdf
http://siteresources.worldbank.org/EXTEVACAPDEV/Resource
s/4585672-
1251481378590/MandE_tools_methods_approaches.pdf
http://www.publicprivatedialogue.org/monitoring_and_evaluatio
n/M&E%20Handbook%20July%2016%202008.pdf
http://siteresources.worldbank.org/INTIMPEVALTK/Resources/
IE_Toolkit_2012.08.21_ENG.pdf
http://www.mhealthknowledge.org/sites/default/files/MAMA_Gl
obal_MEPlan_FINAL_all_0.pdf27431Elements of Project
Evaluation What Are the Elements of Project Evaluation?Power
of Measuring ResultsPurpose of M&E SystemsUSAID
CLAWorld Bank Monitoring and Evaluation (M&E) Are

Synergistic—1World Bank Monitoring and Evaluation (M&E)
Are Synergistic—2Defining Project EvaluationRationale for
M&EEvaluation Elements“Drive out Fear” Edward
Deming27432Evaluation PrinciplesEvaluation Principles:
USAID—1Evaluation Principles: USAID—2Evaluation
Considerations—1Evaluation Considerations—2Hierarchy of
Study DesignsStudy Designs ContinuedHierarchy of Evaluation
Methods?Sample Size and Power DeterminationsCrafting
Evaluation Frameworks—1Crafting Evaluation Frameworks—
2Crafting Evaluation Frameworks—3Performance
IndicatorsConfiguring Performance FrameworksCommon
Project Errors27433Evaluation ComponentsEvaluation
Components—1Evaluation Components—2Facility Performance
MeasuresCommunity and Household SurveysMultidimensional
Evaluation Measures—1Multidimensional Evaluation
Measures—2Global Reference ListWHO 100 Core Health
IndicatorsImpact EvaluationCost-Benefit and Cost-
Effectiveness AnalysisRapid AppraisalRapid Appraisal
Methods—1Rapid Appraisal Methods—2Participatory
EvaluationParticipatory MethodsConstraints of Participatory
EvaluationMixed Methods EvaluationData Quality, Storage
SecurityData Analysis—1Data Analysis—2Presentation and Use
of DataRoad MapReferences
Diarrheal Disease Control & Management
Specifically among Maternal, Neonatal and Children under-five
in Rural-Poor Communities
Presented by:
Problem Statement:
In Ghana, diarrheal diseases are
among the leading causes of

morbidity and mortality in children
under-five, accounting for
approximately 25% of deaths (Binka,
Vermund, & Armah, 2011). While
many children are affected, diarrheal
diseases have a greater presence in rural areas. These diseased
not only
affect overall Maternal, Neonatal, and Child Health (MNCH)
but are a burden
to the economy. The Ghana government spends over US$33
million
annually as a result of diarrheal diseases (Ameyaw, Ameyaw,
& Acheampong,
2017).
Izzy: This is a great problem statement, just a little long. Try to
get it down to 2-3 sentences.
GOAL:
To reduce the mortality and morbidity of diarrheal diseases
among maternal, neonatal, and child populations,
particularly amongst the rural-poor and socially- excluded
communities in Ghana.
Izzy - Fantastic goal!
Objectives & Indicators:
Indicator(s):
CHWs training needs assessment.
Pre & Post-test/survey for CHWs.
CHW training evaluations & outcome reports.
Number of community members who complete training.

Number of targeted trainers.
Objective 1: Community Based Outreach- CHW's
Engage and train community health workers (CHWs) on causes,
control and prevention of diarrheal diseases in maternal,
neonatal and under-five children and ways to educate the
community.
Potential Activities: Distribute literature, host community
classes, health events, train CHWs.
Izzy - your objective 1 starts sounding like an activity (the
doing part to get to your goal). You want to phrase it in a more
SMART format. THink “Increase by X% the number of
CHWS trained in XYZ in X years
Objective 2: Educate Rural-Poor families on treatment options.
Inform rural-poor families of causes andavailable treatment
options for diarrheal diseases and their benefits through
community- based training events.
Indicator(s):
Percentage of MNCH with diarrhea sought care from an
appropriate healthcare provider.
Proportion of MNCH received oral rehydration therapy (ORS)
or recommended home fluids.
Number of community members (families) informed on
available treatments.
Potential Activities: Distribute literature, CHW's host
community classes and training events.
Izzy - These are great indicators but you need to change your
objective to be more SMART
Izzy - Similar to objective 1, this sounds more like an activity.

Indicator(s):
Number of MNCH diarrheal disease cases.
Number of WASH kits distributed.
Household surveys on WASH practices.
Number of WASH community training events hosted.
Percentage of community members who attended the event.
Objective 3: WASH improvement
Provide the knowledge and tools necessary to improve WASH
practices at the household level.
Potential Activities: Host training sessions on WASH practices,
give hygiene kits to all community members with soap, water
tablets, water storage containers, etc and literature teaching
best practices.
Objective 4: Strengthen policies
related to safe disposal of feces.
Collaborate with MOH and in-country health partner to
strengthen national guidelines and policies related to safe
feces disposal management and
the caregiver behaviors.
Indicator(s):
Household surveys on feces disposal practices.
Percentage of communities declared ODF
Proportion of households with a constructed toilet
Potential Activities: Integrate Open defecation free verification
(ODF) protocols and national sanitation policies, strategies, or
monitoring mechanisms.
Implement raising awareness program to change the behavior
of caregivers in safe disposal of child feces
both a community and household level.

US/Ghana Private & Voluntary Organizations (PVOs) Partners:
(USAID, 2017)
(ECHO, 2018)
(MOH_Ghana, 2018) (Path, 2018)
References:
Ameyaw, R., Ameyaw, E., Acheampong, A.O., & et al. (2017).
Diarrhea among Children Under Five Years in Ghana. Global
Journal of Research and Review, 4:2.
Binka, E., Vermund, S.H., & Armah, G.E. (2011). Rotavirus as
a cause of diarrhea among children under 5 years of age in
urban Ghana: Prevalence and serotypes/genotypes. Pediatric
Infectious Disease Journal, 30: 718-730.
USAID. (2017). 2016 VolAg Report: Report of Voluntary
Agencies Engaged in Overseas Relief and Development.
https://www.usaid.gov/sites/default/files/documents/1866/Volag
2016.pdf
ECHO, 2018. Project ECHO® Right Knowledge. Right Place.
Right Time. UNM. https://echo.unm.edu/wp-
content/uploads/2017/09/ECHO_One-Pager_08.17.2017.pdf
MOH, Ghana (2018). Ministry of Health, Ghana. Welcome

Page, http://www.moh.gov.gh/
Path (2018). NGO, About. Diarrheal Diseases.
https://path.org/diarrheal-disease/
USAID MNCH
Diarrheal Disease Ghana
Log Frame &
Monitoring & Evaluation Plan
Dec. 2, 2018
Reduce Morbidity & Mortality of diarrheal diseases in rural-
poor & socially-excluded Ghana communities.
Objectives:Indicators:Means of
Verification:Assumptions:Objective 1:
Strengthen and build capacity of 60% of CHW's and 40% of
rural-poor families with children under-five to reduce the
incidence of Diarrheal Disease in five year project period. % of
population attending WASH training events
# of CHW trainings conducted
# of community trainings conductedSurvey data
Test results
Community members will have knowledge of prevention
methods and be empowered to seek treatment for diarrheal
diseases.
Output 1:
Increased number of CHW’s trained to provide knowledge on
causes, control & prevention of Diarrheal disease. Pre-test &
post tests

Number of CHW’s trained
Test results
Survey data There will be CHWs readily accessible to
community members. Output 2:
Increased amount of literature & materials to educate the
community on causes, control, & prevention of diarrheal
diseases. Number of education materials distributed
Number of CHW lead workshops
Survey data
CHW records CHW’s will be comfortable enough with the
material to teach the material to community
members. Activities:
Establish relationship with current CHW’s and community.
Conduct pre-tests, focus groups, & surveys.
Develop trainings & education materials.
Train CHW’s in the community on diarrheal diseases & on ways
to get information to the community. All to happen at the
beginning of the project, within the first year.
Isabel Slettebak (IS) - You don't need the "..to reduce the
incidence of diarrhea disease". In the next couple of slides if
you see highlighted portions they are not needed
Isabel Slettebak (IS) - This needs to be more specific. What
survey? What test?
Isabel Slettebak (IS) - Assumptions are what do you have to
assume for the activity. Ex: If you are going to be training are
you assuming there is a facility to do the training in?
Isabel Slettebak (IS) - I would reformat this to be included in
the table and matched to its output. Activities should have
indicators, MOV and assumptions

Develop WASH practices in rural-poor communities in Ghana,
providing a 55% increase in access to clean drinking water,
hygiene practices, and sanitation over the 5-year life of the
project.% of population seeking treatment for diarrhea
symptoms.
Proportion of MNCH having received ORS
# of households with a toilet
% of population attending WASH training events
Survey data
Test results Community members will be empowered to seek
treatment for diarrheal diseases. That current systems are
already in place. Output 3:
Improve water systems to create greater access to safe drinking
water. # of homes with presence of harmful bacteria present in
water
# of homes with safe water storage containers
Proportion of homes using a particular water source and # of
diarrheal disease cases.
Proportion of MNCH having received ORS Household surveys
on water practices
Public records
Water test results
Observation of water turbidityCommunities have a central water
system and will be open to a change/improvement. Output 4:
Healthy hygiene and sanitation behaviors are adopted.
% of population attending WASH training events
# of WASH kits distributed

% of households utilizing soap & water to wash their hands
# of households with a toilet Household surveys
Attendance records for community training events
Home visit data
Community is open to change and if given the proper resources
and training will implement the change. Community is willing
to share information regarding their hygiene and sanitation
practices. Activities:
Activities and trainings to promote model hygiene & sanitation
behaviors.
Improvement in latrines and water supply.
Distribute WASH kits to community members.
All to happen mid project, end of year 1 or beginning of year 2
and most of the activities will be ongoing depending on the
communities progress.
Isabel Slettebak (IS [2]) - Great indicators!
Isabel Slettebak (IS [2]) - I would make sure to limit my
indicators to measuring exactly what the output is. How does
ORS have to do directly with improved water systems?
Raise Awareness to increase 70% of rural-poor communities’
knowledge on proper feces disposal and integrate safe disposal
of feces into national policy/guidelines.Proportion of rural
communities with improved feces disposal services
# of community trainings conducted
# of communities with government involvement in feces
disposal Survey data
Test results
Existing records

National dataCommunity members have an existing system of
feces disposal and the ability to change those behaviors. Output
5:
Reduce exposure to diarrheal diseases by improving behaviors
and processes related to feces disposal management in rural-
poor communities. Proportion of houses with a constructed
toilet
% of communities declared ODF Household surveys on feces
disposal practices
Public records Communities will utilize sanitary processes for
waste management. Output 6:
Increase access to healthcare services.Number of health clinics
within 10 mile radius
% of population seeking treatment at health clinics
Proportion of CHWs to population
% of population seeking treatment for diarrhea
symptoms. Survey data
CHW records There will be more healthcare clinics/providers
within an accessible distance from community
members. Activities:
Conduct surveys and focus groups focused on current use of
health system and feces disposal behaviors and practices.
Meet with MoH and country officials to improve feces disposal
management and healthcare services available in these
communities. Ongoing throughout the life of the project.
Isabel Slettebak (IS) - By when?
Isabel Slettebak (IS [2]) - This should be past tense since it is
an output of your activities...you want to have increaseD
Isabel Slettebak (IS [2]) - This sounds more like an activity -
Outputs are what you want to achieve (or what you get out of

your activities)
poor & socially-excluded Ghana communities. Goal &
Objective(s)Performance Indicator(s)Activity Input(s)Data
Collection Methods/Source(s)Frequency & ScheduleResponsible
PartyAssumption(s)Project Goal:
To reduce the mortality and morbidity of diarrheal diseases
among maternal, neonatal, and child populations, particularly
amongst the rural-poor and
socially excluded communities in Ghana.
Reduction in mortality & morbidity of diarrheal diseases among
maternal, neonatal, & child populations.
Diarrheal management & control measures
Direct observations; Feedback from CHW during review
Meetings; Interviews with family heads; Clinic health reports
Years 1-4 (Quarterly)
PMU-Monitoring Committee; Community Leaders; &
Representatives from PATH, ECHO, Healing Waters
International; Africare, & MoH Ghana
Over 90% compliance rate amongst beneficiaries; Timely
disbursement of funds; favorable political will; Minimal
fluctuation rates
Objective 1:
Strengthen and build capacity of 60% of CHW's and 40% of
rural-poor families with children under-five to reduce the
incidence of Diarrheal Disease in five year project period.
CHWs training needs assessment.
Pre & Post-test for CHW’s
CHW training evaluations & outcome reports.
# of community members who complete training.
#r of targeted trainers.
Distribute literature, host community classes, health events,
train CHWs.Direct observations; Feedback from CHW during
review Meetings; Interviews with family heads; Clinic health
reportsYears 1-2
(Bi-weekly)

International; Africare, & MoH GhanaOver 95% compliance
rate amongst beneficiaries; Timely disbursement of funds;
favorable political will; Minimal fluctuation rates
Isabel Slettebak (IS) - Great goal!
poor & socially-excluded Ghana communities. Goal &
Objective(s)Performance Indicator(s)Activity Input(s)Data
Collection Methods/
Source(s)Frequency & ScheduleResponsible
PartyAssumption(s)Objective 2:
Develop WASH practices in rural-poor communities in Ghana,
providing a 55% increase in access to clean drinking water,
hygiene practices, and sanitation over the 5-year life of the
project.Percentage of MNCH with diarrhea sought care from an
appropriate healthcare provider.
Proportion of MNCH received oral rehydration therapy (ORS)
or recommended home fluids.
Number of community members (families) informed on
available treatments.
Number of MNCH diarrheal disease cases.
Number of WASH kits distributed.
Household surveys on WASH practices.Distribute literature,
CHW's host community classes and training events.
Host training sessions on WASH practices, give hygiene kits to
all community members with soap, water tablets, water storage
containers, etc. and literature teaching best practices.Direct
observations; Feedback from CHW during review Meetings;
Interviews with family heads; Clinic health reportsYears 1-2
Bi-weeklyPMU-Monitoring Committee; Community Leaders; &
favorable political will; Minimal fluctuation ratesObjective 3:

Raise Awareness to increase 70% of rural-poor communities’
knowledge on proper feces disposal and integrate safe disposal
of feces into national policy/guidelines.
Household surveys on feces disposal practices.
Percentage of communities declared ODF
Proportion of households with a constructed toilet
.
Integrate Open defecation free verification (ODF) protocols and
national sanitation policies, strategies, or monitoring
mechanisms. Implement raising awareness program to change
the behavior of caregivers in safe disposal of child feces both a
community and household level.Direct observations; Feedback
from CHW during review Meetings; Interviews with family
heads; Clinic health reports Year 3-4
Bi-monthly
favorable political will; Minimal fluctuation rates

Evaluating Integrated Community Health Programs

Recommended

Recommended

More Related Content

Similar to Evaluating Integrated Community Health Programs

Similar to Evaluating Integrated Community Health Programs (20)

More from margaretr5

More from margaretr5 (20)

Recently uploaded

Recently uploaded (20)

Evaluating Integrated Community Health Programs