Training Session 1: Decision-Making Indicators Amber Peterman: Measuring and Analyzing Decision-making in Development Research

1. Measuring and analyzing decision-making in
development research:
An Introduction
Workshop Session #1, Part 1
Peterman, de la O Campos and Olney
December 2, 2014: Bioversity, Rome

2. Empowerment and decision-making: The premise
 Women’s empowerment seen as a development goal in itself (increased
gender equity) – as well as a means to achieving other favorable poverty-related
outcomes.
 Many program designs and development outcomes have been shown to
depend on women’s ability to negotiate favorable allocations of resources
within the household (Doss 2013).
 Thus, women’s empowerment measures are important to measure and
analyze as:
① Program impacts/outcome measures (endline or change over program
period)
② Program moderators/facilitators of program impacts (baseline or initial
conditions/heterogeneous effects).

3. Why decision-making?
 Malhotra and colleagues (2002) review women’s empowerment
indicators in 45 studies, which they characterize as “vast and
interconnected” – across domains (socio-cultural, economic, familial,
legal, political, psychological) at different levels of aggregation
(individual, household, community, institutional).
 The most commonly used definition of women’s individual
empowerment use some version of Kabeer’s (2001) description of
“the expansion in people’s ability to make strategic life choices in a context where
this ability was previously denied to them.”
 Decision-making and autonomy indicators are preferred by many
researchers because they represent direct measures of empowerment
– rather than indirect (proxy measures: e.g. education, earnings, age).

4. Standard decision-making questions in
quantitative household surveys
“Who in your household usually has the final say” …
 Own health
 Own earnings
 Children’s health
 Children’s education
 Small daily household (food) purchases
 Large household (asset) purchases
 Use of family planning
 Collected in the Demographic and Health Surveys and other
large multi-topic surveys
 Typically asked only to women

5. Standard response options
 Respondent herself
 Her partner
 Respondent and partner jointly
 Respondent and others in the
household jointly
 Others in the household
 Could also enter one or more
IDs of household members

6. Variations on the ‘standard module’
 Specificity in domains –aimed at programmatic or culturally specific
important areas of agency and decision-making (Burkina Faso):
 “Can you make the decision to purchase… toiletries such as soap and
toothpaste? Special foods for your children?”
 Asking men or other household members (WEAI)
 Interactive/qualitative assessments of decision-making using ranking
tools (Rwanda)
 Changes in questionnaire wording/design:
 “In an ideal situation, who in your household would make the decision?”
(Ecuador, Yemen).
 To what extent do you feel you can make your own personal decisions
regarding [livestock raising] if you want(ed) to? (WEAI 2.0)

7. Decision-making indicator construction and
analysis
Indicator construction:
 Sole decision-making
 Sole and joint decision-making
 Summation and individual questions versus factor analysis
 Treatment of not applicable domains
 Gap’s in decision-making (if collecting both men and women)
Analysis:
 Multivariate frameworks controlling for household
demographics (impact analysis)
 Interaction terms and stratification (heterogeneous effects)

8. How well are we doing?
 Despite a limited number of qualitative studies examining validation
of questions, there is scant quantitative research examining robustness
of quantitative measures both in survey design and analysis of
indicator construction. Many studies still conflate status (static) with
empowerment (process) (Heckert and Fabric 2013).
 In many cases, although there is evidence that favorable outcomes are
associated with bargaining power, empirical evidence cannot
rigorously identify causality due to study design and data limitations --
it is therefore difficult to identify specific policies that increase
women’s bargaining power in development settings (Doss 2013).

9. Review of programming and impacts on women’s
empowerment (van den Bold and colleagues (2013))
Type of intervention Quantitative
evidence
Qualitative
evidence
1 Conditional Cash Transfers (CCTs) Mixed +
2 Unconditional Cash Transfers (UCTs) Mixed More evidence
needed
3 Microfinance Mixed Mixed
4 Agricultural interventions Mixed/More
evidence needed
Mixed/More
evidence needed
“Hence, while many development initiatives seem to target women specifically, or have
women’s empowerment as one of their objectives, no sufficient body of evidence
overwhelmingly points to success in terms of improving women’s empowerment, or
improving nutrition through women’s empowerment (pp. 29)”

10. Session objectives and organization
 Review the measurement of decision-making indicators and
their use in nutrition and agriculture research (Amber)
 Case studies:
① Quantitative evaluation in Burkina Faso (Deanna)
② Qualitative evaluation in Rwanda (Ana Paula)
 Discuss strengths and weakness of standard quantitative
decision making indicators with application to Ecuador, Uganda
and Yemen (Amber)
 Application to ongoing program evaluation work

11. Works cited
 Doss C. 2013. Intrahousehold Bargaining and Resource Allocation in
Developing Countries. World Bank Research Observer, 28(1).
 Heckert J and MS Fabric. 2013. Improving Data Concerning Women’s
Empowerment in Sub-Saharan Africa. Studies in Family Planning, 44(3): 319-
344.
 Kabeer N. 2001. “Reflections on the measurement of women’s
empowerment.” In Discussing Women’s Empowerment – Theory and
Practice. Sida Studies No. 3. Novum Grafiska AB: Stockholm.
 Malhortra A, Schuler SR and C Boender. 2002. Measuring Women’s
Empowerment as a Variable in International Development. Background
paper prepared for the World Bank Workshop on Poverty and Gender: New
Perspectives.
 van den Bold M, Quisumbing A and S Gillespie. 2013. Women’s
Empowerment and Nutrition: An Evidence Review. International Food
Policy Research Institute (IFPRI) Discussion Paper #01294. Washington DC.

12. Indicator choice and survey design
experiments from cash and food transfer evaluations in
Ecuador, Uganda and Yemen
Workshop Session #1, Part 2
Peterman, Schwab, Roy, Hidrobo & Gilligan (in progress)
IFPRI – Poverty, Health & Nutrition Division
December 2, 2014: Bioversity, Rome

13. Review of literature: Transfers and decision-making
 Review of programs aimed at empowering women and linkages with
nutrition found mixed and thin evidence base for quantitative
measures of CCTs/UCTs impacts on empowerment – however more
promising evidence from qualitative studies (van den Bold et al. 2013).
 Progresa: Qualitatively showed empowerment of women, however
quantitatively only 1 domain out of 5 (spending of own income)
(Adato et al. 2000; Handa et al. 2009).
 Bolsa Familia: Quantitatively showed impacts on 1 out of 8 domains
(contraceptive use) – concentrated in urban areas (de Brauw et al. 2013).
 Zambia Child Grant Program: No quantitative impacts across 9
domains (AIR 2013).
 Kenya Hunger Safety Net Program: Mixed quantitative and
qualitative findings (OPM and IDS 2012).

14. Objectives: Transfers and decision-making in cross-country
transfer evaluations
① Whether relative rankings of decision-making are sensitive to
differences in indicator construction and survey design
experiments.
② Tests correlation between the various decision-making indices
and other proxy (indirect) measures of women’s status
(women’s education and age) or development outcomes
(household dietary diversity and food consumption).
③ Whether the transfer programs had a measurable impact on
decision-making outcomes for women.

15. Introduction: WFP partnership and cross country study
 While the conceptual issues underpinning these issues are well-understood,
there is little rigorous evidence on the relative impacts
of different modalities
 Partnership with the World Food Programme (WFP) to provide
new evidence on impacts and cost effectiveness of alternative
modalities in Bangladesh, Ecuador, Niger, Yemen and
Uganda.
 From 2008 to 2011, number of WFP programs using alternative
modalities increased 10-fold from 5 projects in 2008 to 51 in
2011 ($208 million in programming).
 Target for 2012 was nearly 1/3 of operations in cash, vouchers
and “digital foods.”

16. The Intervention: Context in Ecuador (Hidrobo et al. 2014).
 Cash, food, food voucher
transfers
 Colombian refugees and poor
Ecuadorians
 7 urban centers in the Northern
provinces of Carchi (highland)
and Sucumbíos (lowland)
 6 monthly transfers of $40
 Targeted towards women (76%)
 Conditional on nutrition training

18. The Intervention: Context in Yemen (Schwab 2013)
 Cash and food transfers
 Unconditional seasonal safety net
 Rural areas of Hajjah and Ibb
Governorates
 Bi-monthly transfers for 6
months equal to $25 per month
 Targeted towards head of
household

19. The Intervention: Context in Uganda (Gilligan and Roy 2013)
 Cash and food transfers
 Targeted to households with child
aged 3 – 5 attending UNICEF
sponsored ECD center
 Three rural districts in
Karamojong sub-region (NE)
 $12 every 6 weeks, for 12
months
 Targeted towards primary
caregiver/mother of child

21. Overview: Evaluation designs
 Ecuador: 80 neighborhoods and 145 clusters in 2 stage randomization.
Neighborhoods randomly assigned to treatment or comparison, thereafter,
clusters within treatment neighborhoods randomized to food, cash or
voucher (N = 2,357 households).
 Yemen: 136 food distribution points randomized food or cash. Comparison
household drawn from same areas who were just above the proxy means cut
point for program qualification (N = 3,540 households).
 Uganda: 98 ECD centers randomized cash, food or comparison (N = 2980
households).
 All countries: Before and after household level surveys, additional
facility/community and biomarkers vary by country. Decision-making
modules to one woman per household, interviewed in private.

22. Context: Gender and Development
 Human Development Index (2012): Ecuador (89), Uganda (161) and
Yemen (160) out of 189 ranked countries.
 Gender Inequality Index: Ecuador (83), Uganda (110) and Yemen (148) out
of 148 ranked countries.
 Ecuador: Although equitable frameworks (inheritance, asset ownership etc.)
exist, gender-based violence is high (35% lifetime physical partner violence)
and culture of machismo.
 Yemen: Entrenched gender discrimination, no legal age at marriage, no law
criminalizing spousal rape, restrictions on women’s movement without male
guardians.
 Uganda: Recent progress in legal status of women, however gaps remain in
implementation. Gender-based violence is high (56% lifetime partner
violence) and age at first marriage is 17.9 years (2011 UDHS).

23. Data: Decision-making indicators
 Standard measures: Sole, sole and joint
 Underlying threat points: Who makes the decision, or who
would make the decision in the case of a disagreement or
dispute?
 Division of tasks/preferences: Who would ideally make the
decision?
 Social desirability bias: “There are many women in Uganda who are
able to exert control over decisions in their household and can influence
important aspects of their lives.”

24. Data: Women’s status and Household-level well-being
 Women level:
 Age
 Education (in years)
 Household level:
 Dietary Diversity Index (DDI): Number of unique foods consumed in
the last 7 days (1-47)
 Value of per capita monthly food consumption (includes food
consumed inside and outside of the household)
 Total per capita monthly consumption (food and non-food)

25. Empirical Specification

26. 0.54
Percentage of women's reported sole decision-making
0.72
0.4
0.46
0.61
0.34
0.44
0.47
0.42 0.44
0.4
0.37
0.51
0.67
0.23
0.34
0.56
0.19
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Own work for pay Own health Child's education Child's health Daily food
purchases
Large asset
purchases
accross domains
Ecuador
Yemen
Uganda
Women report making:
 4.5 (out of 9) sole decisions in Ecuador (50%)
 2.5 (out of 6) sole decisions in Uganda and Yemen (42%)
 Highest: Own health, daily food, purchases
 Lowest: Child’s education and large asset purchases

27. 4.47
Comparison of women's decision-making indicators
7.52
6
5.31
9
2.54
3.1
2.43 2.28
6
2.51
4.41
6
10
9
8
7
6
5
4
3
2
1
0
Sole Sole or joint Sole after
disagreement
Ideal decisionmaking Total possible
Ecuador
Yemen
Uganda
 Including jointness increases decision-making in all countries (more in Ecuador and
Uganda)
 Women report higher decision-making in Ecuador after disagreement, lower in
Yemen
 Ideal decision-making is not markedly higher than actual in Ecuador and Yemen

28. Relative rankings of decision-making using factor analysis
Ecuador (N = 1,174) Sole Sole or joint Ideal
After
disagreement
Sole 1.00
Sole or joint 0.31 1.00
Ideal 0.52 0.47 1.00
After disagreement 0.65 0.36 0.43 1.00
alpha statistic 0.91 0.86 0.89 0.90
 Households in Ecuador and Uganda show low correlation between different
constructions of indicators (majority do not exceed 0.50)
 Households in Yemen show higher correlations, however differences still exist (0.74 –
0.89)
 Relative rankings of households may differ based on how questions are asked

29. Results: Associations and impacts
 Associations with women’s status and household well-being
 Age: Positive association
 Education (in years): No association
 Dietary Diversity Index (DDI): Mixed
 Value of per capita monthly food consumption: Positive
association/mixed
 Total per capita monthly consumption: Positive association/mixed
 Impacts of transfers on decision-making
 No impacts of transfer or transfer type in Ecuador or Uganda
 Impacts driven by food transfers in Yemen
 No differential rankings by social desirability introduction in Uganda

30. Summary: What do we know, what can we do better?
 Phrasing matters: Explore the wording of questions which most reflect local
perceptions of how decisions are made, as well as program goals—particularly in local
languages, which often have limited vocabulary for nuances. Formative research is
particularly helpful for both these points when studying a new context with different
cultural and gender norms.
 Ask about the right domains: Specific to the level of influence one might expect the
program to change or depend on for leveraging benefits – specific to context.
 Analysis: Pay attention to response options which reflect possible decision-making
arrangements, and think through how indicators will be constructed. We should not
simply assume that sole decision-making is preferable to joint decision-making,
depending on household structure and power dimensions within the household.
 Qualitative work: Triangulation of evidence may be necessary to uncover certain
domains of empowerment, not able to be captured with quantitative survey methods.
 More research!: We need to continue advancing the frontier of how to most
accurately capture and analyze decision-making and women’s empowerment.

31. Acknowledgements
 In country partners for data collection and survey management: Centro de
Estudios de Población y Desarrollo Social (CEPAR), Yemen Polling Company and
Makerere University.
 IFPRI colleagues including John Hoddinott, Nancy Johnson, Amy Margolies,
Hazel Malprit, Vanessa Moreira and Agnes Quisumbing for helpful
discussions at study conception and contributions through work on the larger
food and cash transfer evaluation. Caroline Guiriec for assistance in
administration of the grant.
 WFP (Rome, Quito, Kampala and Sana’a) for excellent collaboration and
program implementation.
 Funding from the Government of Spain for the impact evaluations and to
the CGIAR Research Program on Agriculture for Nutrition and Health
(ANH) led by IFPRI for the analysis and writing of this paper.

32. Works cited
 American Institutes for Research (AIR) 2013. Zambia’s Child Grant Program: 24-month impact
report. Washington, DC: AIR.
 de Brauw A, Gilligan DO, Hoddinott J and S Roy. 2013. The Impact of Bolsa Familia on Women’s
Decision Making Power. World Development, 59: 487-504.
 Gilligan DO and S Roy. 2013. Resources, stimulation and cognition: How transfer programs and
preschool shape cognitive development in Uganda. Agricultural & Applied Economics
Association’s 2013 AAEA & CAES Joint Annual Meeting: Washington DC.
 Handa S, Peterman A, Davis B and M Stampini. 2009. Opening up Pandora’s Box: The effect of
Gender Targeting and Conditionality on Household Spending Behavior in Mexico’s Progresa
Program. World Development 37(6): 1129-1142.
 Hidrobo M, Hoddinott J, Peterman A, Margolies A, and V Moreira. 2014. Cash, food, or
vouchers? Evidence from a randomized experiment in northern Ecuador. Journal of Development
Economics, 107: 144-156.
 Oxford Policy Management (OPM) and Institute of Development Studies (IDS). 2012. Kenya Hunger Safety
Net Programme: Monitoring and Evaluation Component – Impact Analysis Synthesis Report. Oxford, UK:
Oxford Policy Management: Brighton, UK: IDS.
 Schwab B. 2013. In the form of bread? A randomized comparison of cash and food transfers in
Yemen. Paper presented at the Agricultural & Applied Economics Association’s 2013 AAEA &
CAES Joint Annual Meeting: Washington DC.