Multi-dimensions of Nutrition-related Research to Strengthen Programming: Using Process Evaluation to Strengthen Program Delivery

1. USING PROCESS EVALUATION TO
STRENGTHEN PROGRAM DELIVERY
GHPC May 17, 2016 – Jennifer Nielsen, PhD, Senior Nutrition Advisor

2. WHY PROCESS EVALUATION?

3. • Randomized controlled trials are considered the gold standard
in medicine for evidence of impact of a treatment; increasingly
used for effectiveness of broader public health interventions
• With a highly complex intervention there is need for more
information on the dose and fidelity of delivery to add
plausibility to the evidence (intervention - - - > impact)
• HKI’s Enhanced Homestead Food Production (EHFP) includes
• Training to improve horticultural and husbandry practices
• Behavior change communication to influence nutrition, health, hygiene practices
• Multiple partnerships
• Cascade training structures with three levels
• Gender empowerment/transformation strategies
 In short, many opportunities for losses in both dose and fidelity!
Rationale

4. BCC training
on ENA for
beneficiaries
Outcomes ImpactInputs Process Outputs
HKI, APRG and
governmental
structures
(Ministries of
Health,
Agriculture,
Animals,
Environment, and
the Promotion of
Women, local
authorities and
officials) work
together
Training in
plant and
animal
production
techniques for
master trainers
BCC training
on ENA for
community
level nutrition
trainers
Develop a
training strategy
in animal; and
plant production
techniques
BCC training
on ENA for
master trainers
Develop a
behavior change
communication
(BCC) strategy to
promote Essential
Nutrition Actions
(ENA)
Training in
plant and
animal
production
techniques for
Village Farm
Leaders (VFL)
Village Model
Farms (VMF)
established
Training in
plant and
animal
production
techniques
for
beneficiaries
Establishment
of individual
farms (40
women per
village)
Improved
maternal and
child health
and nutrition
outcomes
Agriculture and
zoological inputs
distributed
Improvements
in small
ruminant and
poultry
production
Improvements
in fruit and
vegetable
production
Improvements in
household
consumption
Increased Income
Beneficiaries
received and
understood
BCC training
on ENA
Improvements in
nutrition and
feeding practices
for children,
pregnant women
and breastfeeding
mothers
Adoption of
agriculture
practices
Women’s
empowerment
improved
Women’s assets
increased
Increased
availability of
micronutrient
rich fruits and
vegetables
Adoption of
ENA
practices by
beneficiaries
Improvements in
care & hygiene
practices for
children,
pregnant women
and breastfeeding
mothers
Beneficiaries
received and
understood
agriculture
training
Increased
availability of
food from
animal origin
 Nutrition Pathway
 Production
Pathway
 Income Pathway
Program Theory - EHFP Burkina

5. AKA…..
5

6. • Program funded by USAID – OFDA (2009-2012)
• IFPRI research lead; HKI implementation lead
• First attempt to adapt EHFP model to Africa
• Implemented as RCT with longitudinal design
following HH with children 3-12 months at
baseline1
• Process evaluation conducted ~ midterm April-
May 2011
1Olney et al (2015, 2016) J. Nutr.
PROCESS EVALUATION IN BURKINA

7. • Mixed methods
– SSI both closed and open ended with
random sample of beneficiaries (n=145)
and controls (n=75)
– KII with purposively selected master
trainers in ag (n=13) and nutrition (n=24)
– KII with community nutrition volunteers:
• older women leaders (n=30)
• health committee members (n=28)
– KII with village farm leaders (n=58)
• Guided by PIP (supplies delivered;
training attended; contents
understood; knowledge gained;
motivation and activities at all levels)
• Explored informants perceptions of
program quality and areas for
improvement
METHODOLOGY

8. • Enumerator training, data collection and
analysis led by IFPRI with significant
involvement of HKI implementing team
• Findings vetted and debated with
implementation team
• Findings also shared with advisory
committees at local and national levels
• Recommendations incorporated, as
feasible, in subsequent work plan
ROLL OUT

9. • Water challenges: demand exceeding supply
– Repaired and built boreholes
– Foot pumps
– Drip irrigation for VMF and HH
– Watering cans and wheelbarrows
– Relocated some VMF
– More drought-resistant crops
• High loss of poultry, low production &
consumption
– Mixed poultry breeds and more local procurement
– Assured vaccine supply
• Low fruit production and consumption
– Mangoes not yet bearing fruit (3-4 years from)
– Papayas bear for 8 months but ~50% died
FINDINGS: PRODUCTION & INCOME PATHWAYS

10. • Leakage from Master Trainers to
Volunteers to Beneficiaries
– Some topics in particular (rich
complementary foods, feeding sick child,
anemia control, women’s nutrition)
– Refresher training on key topics
– Focus messages (egg consumption)
– Training in facilitation skills
– Supportive supervision
• Motivation of volunteers mixed
– Public recognition
– Small gifts
• Home visits less than expected
– Emphasize group discussions
FINDINGS: KNOWLEDGE - CONSUMPTION PATHWAY

11. • Lag times at multiple stages
reduced exposure
– From award to beginning of
implementation (6 months)
• Training of Master Trainers and delivery of
inputs
• Planning of research
– Between baseline and VMF production +
nutrition BCC (6 months)
• Training of VMF then HH
• Staple (rainy season) production
• ENA training
• HH gardens year 2
– Horticulture production on off (dry)
season
FINDINGS: LOGISTICS
• Seasonality constraint (endline = baseline) allowed only 2 years
of implementation

12. • Tremendous enthusiasm of
communities and local partners
• Within intervention communities
– “My husband thinks that my work at the VMF
is contributing to improved health in our
household”
– “I have a family garden to cultivate
vegetables and meet the nutritional needs of
my household”
• Spillover doubled population reach;
demand from control communities
TESTIMONIALS

13. • Research findings valuable but come at cost to
implementation
– Staff time to support logistics
– Time burden on beneficiaries and controls
• Not all problems can be solved
– Water and soil constraints
– Weaknesses in larger input delivery systems
– Intrinsic motivation not universal (also VMF vs. ENA)
– Low education level of community volunteers
• Complexity of model
– Multisectorality of contents and partners
– ENA (and now EHA) dense with messages and behaviors to
change
CHALLENGES IN PROCESS EVALUATION

14. • Findings informed strengthened
program delivery
• Process gave voice to staff and
beneficiaries
• Informed changes to model for
subsequent phase
– Timing/design of research
– Water management plans
– Incentive strategies
– Reinforced WASH and malaria
control components
– Test LNS to fill nutrient gaps
• Effort is high; returns are higher
RETURNS

15. MERCI
15
“We could never learn to be brave and patient if there were only joy in the world.”
– Helen Keller