Assaad Bridging Evidence & Policy Talk.pptx

1. Lessons from Bab-Amal Impact Evaluation
Ragui Assaad
Professor, University of Minnesota & co-Principal Investigator, Bab Amal
Launch Event: Impact Evaluation of Social
Protection Programs
Bridging Evidence and Policy (BEP) Seminar Series
Co-organized by Egyptian Food Bank (EFB), IFPRI,
and Sawiris Foundation for Social Development
(SFSD)
MAY 7, 2023 - 9:00AM TO 1:00PM EEST

2. • The Graduation Approach: An Effective Big Push
Poverty Reduction Program
• Bab Amal is a pilot program adapting the
internationally recognized Graduation approach
and targeting 2,400 extreme poor households in
Assiut and Sohag governorates, implemented by
Sawiris Foundation in cooperation with two
development NGOs.
• Consists of a comprehensive, time-bound, and
sequenced set of interventions that aim to
graduate people from ultra-poverty into
sustainable livelihoods
Introduction

3. Bab Amal: Identifying Key Players
BRAC
Sawiris Foundation
for Social Development
Assiut “Giving without Limits”
Sohag “Human Development
Egyptian Association”
Partners Principal Investigators

4. Bab Amal: Theory of Change
Intervention 1
Full Cost
Unrestricted
Intervention 2
Full Cost
Restricted
Intervention 3
Half Cost - Unrestricted
Comparison
No
intervention
Social
Protection
Social
Empowerment
Financial
Inclusion
Livelihood
Promotion
Productive asset transfer - EGP 11,000
Technical and business training
Consumption support EGP 400 (8 months)
Linkages to existing government services
Emergency Health Fund
Saving support (biweekly saving meetings)
Financial literacy training
Biweekly household visits (18 months)
Biweekly group visits (coaching and
saving group meetings) – 18 months
Consumption support EGP 200 (8 months)
Linkages to existing government services
Emergency Health Fund
Productive asset transfer - EGP 5,500
Technical and business training
Monthly household visits (18 months)
Monthly group visits (coaching and
saving group meetings) – 18 months
Saving support (monthly saving meetings)
Financial literacy training
Graduation
Pillars

5. Intervention Design
Ultra-poor
Households
Intervention 1
Full program –
unrestricted
choice of
assets
Intervention 2
Full program –
restricted
(to female
beneficiaries)
Intervention 3
Half-cost
program –
unrestricted
set of assets
Comparison
No
intervention
Policy relevant

6. Motivation and research questions
1. How well does the standard “Graduation Program” work in Egypt?
1. Can prioritizing the targeting of women expand the impact of the
program and result in greater empowerment?
Intervention 1: Full cost – Unrestricted vs. Comparison group
Intervention 2: Full cost – Restricted to women vs. Comparison
group

7. 3. What variations could make the program more cost-effective and/or
sustainable?
3. Compare Interventions 1, 2 and 3 to each other: explore cost-
effectiveness aspects
3. Is there potential for scale up?
Motivation and research questions
Intervention 3: Half cost vs. Comparison group

8. 2018 Dec 2018- Feb 2019
Geographic
targeting
Survey
Targeting
Jun- Aug 2019 Aug 2019 Oct 2019
May 2020
Aug 2021-Oct 2022
Jan-Dec 2023
Research,
publication, and
dissemination
Mar-Aug 2019
Understanding Context:
Needs Assessment
Market Assessment
Design Evaluation
Targeting
“Bab Amal” Timeline
Baseline Survey
Information Sessions Randomization Implementation
Endline Survey Midline Survey
Covid-19
Phone Survey

9. Hybrid Targeting
Step 1
Geographic Targeting
- Using Egypt’s poverty
map to identify poorest
governorates (Sohag
and Assiut) and villages
- Split villages to sub
areas called
agglomerations to be
used as unit of
randomization
Step 2
Household Targeting
● Hybrid, combination of
census data on
households identified in
step 1 to generate
proxy means testing
scores
Step 3
Community-based
Targeting
● Community- based
targeting via
communities leaders
and representatives

10. Eliciting preferences from households and Information
session
Baseline survey (adult section)
• Select a project for each of the 3 treatment
arms and designate two adults for each
project
Information session:
• Conducted by the NGOs facilitators
for each household on:
• Pros and cons of each project; pre-
requisites; risks; experience
• The household has 2 days for reflection
to decide which member of the
household will manage the asset
under each treatment arm
• Randomization into the three
intervention groups and the
comparison group takes place after
the information sessions have been
completed

11. How Choices Differed by Treatment Condition
● Majority of assets chosen were
Livestock Assets (goats and sheep
for fattening or milking)
● Examples of Non-Livestock assets
chosen included: Sewing machine,
Dairy products, Ready made
garments trade, grocery, etc..
● 88% of households chose a woman
in the unrestricted scenario

12. Sample Allocation
We randomize 3469 households
into four groups

13. Midline Survey
• Household questionnaire
– Demographics; Education; Labor market status; Agricultural activities; Non-agricultural
activities; Remittances and transfers; Savings; Health/covid-19; Consumption; Food
security; Shocks
• Individual questionnaire
– Answered by the designee (person who was designated to be the beneficiary)
– Aspirations for children; decision making; time use; women mobility; gender roles;
physical and mental health; livelihood; finances
• Beliefs module
– Women’s beliefs related to their hypothetic participation in different economic activities
– Answers are given in a range of probabilities (1 to 20) using beans and pictograms
– 8 female economic activities were identified (focus group discussion in April 2022)
Data collection was conducted between August 2021 and October 2022

14. Livestock ownership
• All intervention groups own
a larger number of animals
than the comparison group
• Similar results for the
restricted and unrestricted
intervention groups
• Half cost: the additional
number of animals
compared to the
comparison group is greater
than half of the additional
number of animals of the
two full cost groups

15. Revenue from Livestock
• Revenue of intervention
groups related to livestock
is more than double that of
the comparison group
(multiplied by 6 in the case
of medium size livestock
and multiplied by almost 4
in the case of the large size
livestock)
• Similar results for the
restricted and unrestricted
intervention groups

16. Household Income
● All intervention groups have
a greater income than the
comparison group
● Full-cost groups: the
income increase ranges
between 11% (unrestricted)
and 18% (restricted to
women)
● Half cost: the additional
income compared to the
comparison group (+9%) is
equal or greater than half
of the additional income of
the two full-cost groups

17. Female employment
• Female employment of
intervention groups is more
than double that of the
comparison group
• Full-cost unrestricted and
half-cost groups have
similar results: +14
percentage points
• Full-cost restricted to
women group: larger
impact on female
employment (+17
percentage points)

18. Food insecurity
• Food insecurity index
combines several indicators
on quantity, frequency,
quality, variety of food and
hunger over the past 12
months
• The three intervention
groups suffer less from food
insecurity than the
comparison group.
• Similar results for the two
full-cost groups
• Half-cost group: the
impact is equal to half that
of the full-cost groups

19. Lessons Learned and Next Steps

20. Lessons learned for potential scale-up
• Bab-Amal is an effective approach to reduce poverty and potential put
the poor on a sustainable path to graduate from poverty
• Use of agglomeration level as a unit of targeting and randomization is
adequate to obtain good statistical power, practical, and cost-effective
• Targeting using proxy means score with community leader validation
effective
• Having a national proxy mean score based on detailed administrative data
would be extremely helpful to target social protection projects
• Restricted versus Unrestricted arms: preliminary results show limited
differences
● Given that ”livestock” is the most likely choice of asset, restricting receipt of asset to
women may not be necessary given existing gender divisions of labor in rural Egypt

21. • Half cost versus full cost:
- Seems like half-cost is about half as effective
- Sometimes more (livestock ownership), sometimes less.
• Long-term follow-up necessary to determine size of necessary intervention
• poverty traps may cause households to fall back into poverty if economic push is not
large enough
• cannot yet conclude that half-cost is sufficient (or even full cost for that matter)
• endline (36-40 months after transfer of asset) will provide ore conclusive information
about medium-term effects
• Even longer-term follow-up would be highly desirable
• Deeper understanding of beliefs and expectations about women’s participation in
economic activity will allow us to identify intervention can can empower women
economically while accounting for local norms and beliefs and women’s existing
roles
21
Lessons learned for potential scale-up

22. Thank you