Global Workshop on Nutrition-sensitive Fish Agri-food Systems 5-8 December 2017, Siem Reap, Cambodia

1. LESSONS FROM THE RAIN PROJECT
Process and impact evaluation design of a
nutrition-sensitive agricultural intervention
Jody Harris
Global workshop on nutrition-sensitive fish
agri-food systems. Cambodia, Dec 2017

2. RAIN
Projec
t
Agriculture
Vegetables,
fruits, small
animals
Coordinatio
n
&
Alignment
Nutrition
and Health
Behaviour
Change
CommunicationGender
Empowerment
,
mainstreamin
g
HIV &
AIDS
Mainstreamin
g
Monitoring
, Learning
&
Evaluation
RAIN PROJECT
2

3. RAIN evaluation
• Repeated cross-
sectional impact
evaluation
• Randomized at Ward
and cluster level
• Multi-module
questionnaire based on
UNICEF framework
• Two rounds of mixed-
method process
evaluation
Cross-sectional baseline survey (2011)
Mumbwa District: 6 wards
13 Clusters
(n=1000 HHs)
Agriculture Only
13 Clusters
(n=1000 HHs)
Agriculture + Health
2 comparison wards4 intervention wards
26 Clusters
R
R
Cross-sectional impact evaluation survey (2015)
13 Clusters
(n=1000 HHs)
Agriculture Only
13 Clusters
(n=1000 HHs)
Agriculture + Health
15 Clusters
(n=1000 HHs)
Comparison
Process evaluation (2012 – 2014)
15 Clusters
15 Clusters
(n=1000 HHs)
Comparison

4. ln:rer...ed
COr'*"'IQlO'I (JI
tu:t!Ct"C"'tCh foods
• : - RAlN°"'""""'"'- 1-..........;_. ....-..-• • • • • • • • • •, • H e .IU'Im •er :
: :..! ' --;- - "- - - _ , -_ - - -: • • .....J'•••••• • • •
·• . .•+; MMIU'IMastfl :
:... - . _:
·.··-········ .• Cl1Vsr«rutect _.., !
! U'...w:dt • td rd W s) ,
'
... ..-,·........
! CHVsritt..,..._ll"d :
: .....,_.... :
1 ··
...
-- . -
..-•SMISrtcNC.ed
'rwiecs ( • r drd' es t
•
Womrsfs fQ.P
-•wcmen m«ld
... . .. 1.. . . .. . .
SMfSf SbllS f
--· f-oO
S M f s rTK!det
-
.........._........
, WO'Tltl'lI C •e
:
l+- - - - ·-···-······tio
:: thb"O#ltelte :
"'°Slurts :' • • • • • •1• • ... ... .
.
• •-
.•.•..
.....
!··· ··· ··· -:- - -'
•
...
:..- - - - - --1
.•
}- - - - -
Ctw'ct 1nre5Cl.lc
cconcrooed°"womenj
SMf s detM!r itOJP
• lTloes.SllllfC...,
l"JO'l' visits
- -
W o m e n aofiSn
srnau ar
pr""-<tlon
L
- -
O . .cinwanM '
lnfluer.:e/ dee.
makfrc poHs
I
5

5. Impact evaluation: Objectives
• To assess the impact of the two different RAIN intervention packages on
stunting among children aged 24 to 59 months
• To assess the impact of the different RAIN packages of interventions on:
• Availability of and access to a year-round supply of diverse and
micronutrient-rich plant and animal source foods at household level
• Infant and young child feeding (IYCF) indicators among children 0-23
months of age
• Preventive and curative health practices and nutrition knowledge
among mothers
5

6. Process evaluation
Aim: To open the ‘black box’…
• To assess which elements of the program worked, which didn’t, and
which contributed in which ways to the ultimate success or failure of the
project, and why.
1. Development of Program Impact Pathway (PIP) diagram from detailed RAIN
project operational manual.
2. Identification of key domains within the PIP on which to focus assessment.
3. Identification of specific questions, and respondent groups.
4. Data collection and analysis- qualitative and quantitative
6

7. Management and partnerships
Training
Motivation, knowledge, performance
Frontline service delivery
Program exposure
Program adoption
Outcome and impact

8. INPUTSPROCESS
SMFs deliver group
ag messaging and
home visits
SMFs establish model
farms
Women establish
small animal
production
OUTCOMESOUTPUTSIMPACT
Boreholes assessed
RAIN partnerships
Agriculture master
trainers identified
Health master
trainers identified
Agriculture inputs
procured
Women’s groups
established
Agriculture master
trainers trained
Health master
trainers trained
SMFs receive
agricultural inputs
Training manuals and
tools created
SMFs recruited and
trained (+ refreshers)
CHVs recruited and
trained (+ refreshers)
CHVs deliver group
BCC and home visits
Women’s groups
receive agricultural
inputs
Women acquire
resources
Women establish
gardens
Change in stunting rates
CHVs retain skills and
knowledge
SMFs retain skills and
knowledge
Change in women’s
influence/ decision-
making power
Women attend
groups
Gender master
trainers identified
Women acquire ag
knowledge and skills
Women acquire
health knowledge
and skills
Increased
consumption of
nutrient-rich foods
Improved health
knowledge and
practices
Improved IYFC
knowledge and
practices
Change in resources
(controlled by
women)
Change in women’s
time use
Gender master
trainers trained
Gender trainers
deliver
messaging to
groups
Change in
gender
attitudes
Introduction of
labor saving
technology
(stoves)
Reduced burden on
women
Borehole
rehabilitation
undertaken
Improved access to
water
CDFs
facilitate
frontline
work
Increased fruit and
vegetable production
Increased animal
production
Improved post-
harvest management

9. Production
diversity
Women’s
Empowerme
nt in Ag
Index
Household
food
security
Individual
diet diversity
Target food
consumptionAnthropometry
/ biomarkers
Knowledge /
attitudes
Time use
Infant and
young child
feeding
Control and
influence
Food system
interactions
Water
practices
Food safety
and
zoonoses
Food
preparation

10. Conclusions
• Rigorous evaluation can provide evidence not only on what works, but also on
why it works (or doesn’t work)
• Basing evaluations on clear project theories of change provides a logical basis
for understanding different project aspects
• We have good methods and indicators for some steps on the pathways
between agriculture and nutrition, but some require further work.
• Agriculture-for-nutrition projects should focus on diets rather than nutrition
• Any development project must be tailored to context

11. Thank youl