High Flyers Revisited: Using Machine Learning to Understand Who Benefits the Most from a Cash Transfer by Gelson Tembo

1. High Flyers Revisited:
Using Machine Learning
to Understand who
Benefits the most from a
Cash Transfer
Gelson Tembo
Palm Associates and University of Zambia
(on behalf of the study team)

2. Study Overview
● Use data from the impact evaluation of three government unconditional cash
transfer programs (UCT) to assess
○ Who benefits the most from the interventions? and
○ Who benefits the least from the interventions
○ Zambia CGP; Zimbabwe HSCT; Malawi SCTP
● Use new Machine Learning Algorithms (MLA) ‘causal trees and causal forests’
to identify initial household and contextual characteristics that predict
heterogeneous treatment effects (‘highflyers’)
○ ‘Causal trees and causal forests’
● Once ‘highflyers’ have been identified:
○ 1) What are their characteristics?
○ 2) What actions did they take with the cash to become highflyers?

3. A very quick overview of the MLA that we use
1. We provide the algorithm with a list of 30+ indicators and ask it to use these
indicators to group households based on the impact of the programme on
consumption per capita
2. Algorithm scans all the combinations of these 30+ indicators and groups
households with similar ‘impacts’
3. For example, it may find that MHHs under the age of 40 living close to a market
all have the same level of impact—this would be one group; another group
with similar levels of impact may be FHHs who are widowed, and so on
4. The power of the approach is that it is data driven—it identifies groups based
on the indicators that we feed it (rather than us deciding beforehand what the
groups might look like)

4. Example based on Zambian data
-The number of times a particular variable is
used to split the sample
-Here we see productive assets is important
-But age of head and household size are not

5. -Here we see an example of the groupings or ‘causal tree’
in the case of Malawi.
-All households are put into the
boxes (‘nodes’) at the bottom of each branch (purple)
-Households in each box have the same or similar impacts

6. Here we see the distribution of impacts on
consumption in each country
Low flyers
Low flyers
Low flyers
High flyers
High flyers
High flyers

7. -So what are the characteristics of low- and high-flyers?
-Lower dependency ratio, smaller household size
-Worse productive profile, less debt
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Size Dep. Ratio Debt Livestock Prod assets
Zimbabwe
Lowflyers Highflyers

8. -Exact same pattern in Zambia and Malawi!
-Lower dependency ratio, smaller household size
-Worse productive profile, less debt (except in Malawi)
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Size Dep. Ratio Debt Livestock Prod
assets
Zambia
Lowflyers Highflyers
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Size Dep. Ratio Debt Livestock Prod
assets
Malawi
Lowflyers Highflyers

9. Next step – post-treatment behaviors
• Highflyers do seem to have a somewhat unique profile compared to low flyers
• Similar pattern across the three countries – promising
• Do the post-treatment behaviors provide any clues about pathways out of
poverty?
• ‘No strings cash’ provides a nice set-up to examine this
• Are there patterns in the (combination of) activities pursued by highflyers?
• Look at behaviors separately—maybe not so revealing as the combinations
• Use a multidimensional MLA called k-means clustering to look for groups of behaviors

10. -Highflyers show larger improvements in livestock, income
generation, NFE, and debt reduction relative to lowflyers
-Clear difference in level of livelihood diversification
-0.4
-0.2
0
0.2
0.4
0.6
0.8
Livestock Income-Revenue Prod Assets Finance & Debt NFE
Zambia Post-Treatment Behavior
Lowflyers Highflyers

11. -Even among labour-constrained households in Malawi, larger
improvements in productive assets, NFE, livestock and income generation
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
Livestock Income-Revenue Prod Assets Finance & Debt NFE
Malawi Post-Treatment Behaviors
Lowflyers Highflyers

12. -In Zimbabwe where households also labour-constrained, pattern is similar but not
as strong
-Improvements in livestock and productive assets but not in other areas
-Operational hiccups in ZIM may explain this (irregular payments)
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
Livestock Income-Revenue Prod Assets Finance & Debt NFE
Zimbabwe Post-Treatment Behaviors
Lowflyers Highflyers

13. How can this help in the quest for ‘graduation’?
• Highflyers have somewhat unique profile at baseline
• Smaller households, fewer kids, lower dependency ratio – more able-bodied members
• Low debt is important characteristic – more space to use cash for productive activity
• Contextual/environmental factors ambiguous (results not presented here)
• Post-treatment outcomes in productive domains much higher for highflyers
• Zambia CGP: Strongest effects, NFE, revenue from crops sales implies more market
engagement
• Malawi SCTP: Next strongest effects, labour-constrained households may not have same
capacity to invest
• Zimbabwe HSCT: Weakest patterns, labour-constrained households and irregular payments
may explain this

14. Programmatic implications for Cash+ initiatives
• Additional ‘plus’ components, especially productive ones, must carefully
consider profile of households—do they have the capacity to invest?
• Debt position is very important, consistent with theory of poverty traps
• VCLs and other initiatives important to help households manage liquidity constraints and
shocks, and escape debt traps
• Core CT programme must provide a minimum base for households
• Adequate transfer value; regular and predictable payments
• Without this base, ‘plus’ activities may not be as successful

15. Extra Slides

16. Define top and bottom 10 percent of CATEs as ‘high’ and ‘low’ flyers:
Any difference in baseline characteristics?
Low flyers
High flyers
Low flyers
High flyers

17. Sample sizes and survey years (first year is pre-treatment)
Treatment Control Design Survey years
Malawi SCTP 1,730 1,800 RCT 2013, 2014, 2015
Zambia CGP 1,260 1,252 RCT 2010, 2012, 2013,
2014
Zimbabwe HSCT 1,029 1,034 Geographically matched
delayed entry eligibles
2013, 2014, 2017

18. Domains and illustrative indicators; used domain
indices and individual indicators (30+)
Domain Specific indicators
Assets Livestock; productive assets; domestic assets
Finance/debt Savings; new loans; value of debt;
Income/revenue Value of harvest; crop sales; NFE; livestock revenue;
spending on inputs
Psychological state Subjective well-being; patience; future optimism
Environment Rainfall; Palmer dryness index; distance to district
capital; land use (% agriculture)
Household Age, sex, school of head; household size