Growth Week 2011: Ideas for Growth Session 5 - Governance, Accountability and Political Economy
Reshaping institutions: Evidence on external aid and local collective action Katherine Casey, Stanford UniversityRachel Glennerster, Abudl Latif Jameel Poverty Action Lab, MIT Edward Miguel, University of California, Berkeley September 2011
Motivation (1)• Many scholars agree that institutions are important determinants of economic development and that a history or hierarchical and noncompetitive institutions are associated with poor performance. – E.g. Banerjee and Iyer, 2004• However, there is limited consensus on exactly what the “right” institutions are, and even less evidence on how to “improve” existing institutions in poor countries.• Measuring institutional performance is challenging: --Subjective measures are prone to “halo effects” (Olken 2009) -- Institutions are multi-faceted, leaving open the risk of data mining or “cherry-picking” of results consistent with prior beliefs. -- Institutions are themselves affected by economic performance IGC, September 2011 Reshaping Institutions 2
Motivation (2)• A key question is whether it is possible for foreign aid donors to transform institutions in less developed countries? (Is it even desirable?)• Among donors today, arguably the most popular strategy to promote accountability, competence and inclusion of under- represented groups in local government institutions is “community driven development” (CDD). Billions of dollars in donor funding per year.• Influential evidence from India that “imposing” greater representation of minority groups can create a demonstration effect and empower minorities in the longer term -- Beamen et al 2009 IGC, September 2011 Reshaping Institutions 3
Motivation (3)• In this paper we evaluate one attempt to transform local institutions in post-war Sierra Leone. We exploit a randomized experiment to assess CDD impacts on local public goods and local institutions.• We develop new, objective institutional performance measures, and employ a pre-analysis plan to eliminate data mining.• We show that without the preanalysis plan we could have cherry picked data to tell two very different stories: that CDD improved institutions, and that it undermined social capital. IGC, September 2011 Reshaping Institutions 4
Why might Sierra Leone’s institutions warrant reform?• Legacy of bad governance and corruption in the formal system – Brief period of post-Independence (1961) stability and growth – President Siaka Stevens abolished local government (1972) and banned rival political parties (1978), abysmal public services – Brutal civil war (1991-2002)• The traditional system is (also) dominated by elder male elites – 149 Paramount Chiefs rule for life; come from hereditary ruling houses; and control land, labor and the judiciary outside the capital – Women are not eligible for chieftaincy in most of the country• Scholars point to seeds of unrest in social divisions and inequalities – Anger at the failings of the corrupt ruling regime (Richards 1996) – Inequality between chief and subject, including capricious fines, coerced labor and unpopular land allocations (Keen 2003) – Disenfranchisement of women and youth from decision-making IGC, September 2011 Reshaping Institutions 5
What does CDD aim to do?• Financial grants for local public goods, small enterprise development – The "GoBifo" Project ("Move Forward") we study in Sierra Leone gave $4,667 to communities in 3 tranches (~$100 per household)• Training and facilitation to build durable local collective action capacity (6 months of intensive contact spread out over 4 years) – Forms a representative Village Development Committee to promote democratic decision-making – Helps communities agree on a medium-term development plan – Establishes bank accounts and transparent accounting procedures• Requirements to increase participation of marginalized groups – Women were co-signatories on the community bank accounts – Recorded how actively women, youths (18-35 years) participated – Women and youths managed own projects, e.g. labor groups IGC, September 2011 Reshaping Institutions 6
Elements of the study• Randomized experiment produces rigorous evidence on causal impacts with a relatively large sample (236 villages, 2,832 households), and extended time frame (2005 to 2009).• Combines survey data with three "structured community activities" (SCAs) that unobtrusively observe communities post-program: – responding to a matching grant opportunity, – making a communal decision, – managing a public asset.• Hypothesis document agreed in 2004 of key objectives of program• Analysis follows a pre-analysis plan based on hypothesis document, to avoid data mining. IGC, September 2011 Reshaping Institutions 7
U.C. Berkeley September 2011 Reshaping Institutions 8
Local public goods construction projects• The distribution of community projects by sector was: – Infrastructure (43%) - e.g., community centers, primary schools – Agriculture/livestock (40%) - e.g., seed multiplication, goats – Skills training, small business (17%) - e.g., carpentry, soap-making IGC, September 2011 Reshaping Institutions 10
Data collection• Household survey panel (male, female, youth, non-youth respondents)• Field supervisor direct assessments of local public goods quality.• Village focus group discussions with local leaders.• A novel component - structured community activities (SCAs): – Matching grant: communities received six vouchers that could be redeemed with a co-pay at a local building materials store (max value $300). A direct measure of collective action capacity. – Communal choice: communities were presented with two equally valued assets (batteries vs. salt) and enumerators observed ensuing deliberations, recording the number of male/female and youth/elder speakers as measures of participation and influence. – Managing an asset: communities were given a large tarpaulin, useful as an agricultural drying floor or roofing material. Focus on elite capture in a surprise follow-up visit 5 months later. IGC, September 2011 Reshaping Institutions 11
Pre-analysis plan• Tying the researcher’s hands with a pre-analysis plan limits selective presentation (“cherry-picking”) of results (Leamer 1974, 1983), and produces appropriately sized statistical tests (Anderson 2008).• Randomized experiments may not lead to more reliable results, if not all research results are published. Registering pre-analysis plans in a public archive can also limit publication bias.• Over the past decade, pre-analysis plans for experimental studies (and some observational studies) have become standard in medical research, but this is one of the first economics studies with one. IGC, September 2011 Reshaping Institutions 12
Pre-analysis plan• Before the program began in 2005, the research and project teams together agreed to a set of hypotheses about GoBifo impacts.• Before analyzing any endline data, we submitted the exact list of outcome and explanatory variables under each hypothesis, and econometric specifications, to the J-PAL project registry. – Main plan submitted in August 2009 and a supplement concerning outcomes from the second follow-up survey in March 2010, during the respective periods of data entry and reconciliation.• Project impacts are determined by the mean treatment effect across all outcomes under a given hypothesis (Kling and Leibman 2004). – Results for all 318 pre-specified outcomes in web appendix. IGC, September 2011 Reshaping Institutions 13
Econometric specifications• Basic model for outcomes with post-program data only: Yc = β0 + β1Tc + Xc′Γ + Wc′Π + εc – Yc is outcome in community c (HH data averaged by village) – Tc is an indicator for GoBifo treatment – Xc is a vector of community-level controls (results are robust to their exclusion); Wc are ward fixed effects – εc is an idiosyncratic error term• Panel specification where longitudinal data available: Yc = β0 + β1(Tc * POSTt) + β2Tc + β3POSTc + Xc′Γ + Wc′Π + εc – POSTt is an indicator for the post-program period (2009) IGC, September 2011 Reshaping Institutions 14
Hypothesized CDD impacts• 1) Participation requirements on women and youth aim to decrease their disutility of participation – Requirements automatically translate into higher participation during project implementation• 2) Grants subsidize construction and training, reduce the marginal costs of public goods and start up of community enterprises during the program, leading to more construction and economic activity• 3) The increase in community participation, plus establishment of committees, plans and bank accounts, reduces the fixed coordination costs of achieving collective action – If the organizing institutions are durable, the beneficial effects should persist into the post-program period – If women and youths learn-by-doing or their participation exerts positive demonstration effects on others, shifting social norms, this could trigger a sustained improvement in participation IGC, September 2011 Reshaping Institutions 15
Overview of results• Outcome family A: Positive infrastructure or hardware and activity – Village-level structures and tools to manage development projects were established (e.g. bank accounts) – Finances were disbursed with little leakage (<13% discrepancies) – Increases in the stock and quality of local public goods – Increases in household assets and village-level market activity• Outcome family B: Zero impact on “software” / “institutions” – No impacts on participation in decision-making – No sustained increase in collective action capacity – No change in the “voice” of women and young men – Apparent “capture” of new organizations by chiefly authorities – Example of communal farms: established but low participation IGC, September 2011 Reshaping Institutions 16
Table 1: Baseline (2005) Comparison between Treatment and Control Communities Baseline mean T-C N for controls difference at baseline (1) (2) (3)Panel A: Community CharacteristicsTotal households per community 46.76 0.30 236 (3.67)Distance to nearest motorable road in miles 2.99 -0.32 236 (0.36)Index of war exposure (range 0 to 1) 0.68 -0.01 236 (0.02)Historical legacy of domestic slavery (range 0 to 1) 0.36 0.03 236 (0.06)Average respondent years of education 1.65 0.11 235 (0.13)Panel B: Selected Outcomes from "Hardware" Family AProportion of communities with a Village development committee (VDC) 0.55 0.06 232 (0.06)Proportion visited by Ward Development Committee (WDC) member in past year 0.15 -0.01 228 (0.05)Proportion of communities with a functional primary school 0.41 0.08 230 (0.06)Average household asset score -0.06 0.11 235 (0.08)Proportion of communities with any petty traders 0.54 -0.01 226 (0.06)Panel C: Selected Outcomes from "Software" Family BRespondent agrees that chiefdom officials can be trusted 0.66 -0.01 235 (0.02)Respondent agrees that Local Councillors can be trusted 0.61 0.00 235 (0.02)Respondent is a member of credit / savings group 0.25 -0.03 235 17 (0.02)
Table 2: Summary of GoBifo Program Impacts by Research Hypothesis and Outcome FamilyHypotheses by family GoBifo Mean Effect (std. error)Family A: Development Infrastructure or "Hardware" EffectsMean Effect for Family A (Hypotheses 1, 2 and 3; 37 total outcomes) 0.352** (0.030)H1: GoBifo creates functional development committees (7 outcomes) 0.687** (0.062)H2: GoBifo increases the quality and quantity of local public services infrastructure (16 outcomes) 0.164** (0.040)H3: GoBifo improves general economic welfare (14 outcomes) 0.399** (0.047) IGC, September 2011 Reshaping Institutions 18
Table 3: Illustrative Treatment Effect Estimates for Selected Family A Outcome MeasuresRow Outcome variable Mean in Treatment Standard N Specification Controls Effect Error Panel A: Hypothesis 1 - Full Sample Outcomes 1 Village development committee 0.458 0.341** (0.077) 467 Panel 2 Visit by WDC member 0.212 0.156* (0.070) 462 Panel 3 Village development plan 0.617 0.296** (0.048) 221 Cross section 4 Community bank account 0.081 0.706** (0.045) 226 Cross section Panel C: Hypothesis 2 - Full Sample Outcomes10 Existence of functional local Primary School 0.462 -0.007 (0.050) 464 Panel11 public good in the community: Grain drying floor 0.237 0.104 (0.066) 459 Panel12 Traditional midwife post 0.079 0.175** (0.035) 235 Cross section13 Latrine 0.462 0.210** (0.059) 234 Cross section14 Community center 0.212 0.241** (0.063) 469 Panel15 Community took a proposal to an NGO or donor for funding 0.292 -0.156+ (0.081) 460 Panel Panel E: Hypothesis 3 - Full Sample Outcomes25 Total petty traders in village 2.432 0.719* (0.344) 225 Cross section26 Total goods on sale of 10 4.449 0.560* (0.240) 236 Cross section27 Household asset score -0.170 0.212* (0.090) 471 Panel28 Household asset quintile 2.835 0.158+ (0.094) 471 Panel29 Attended skills training 0.061 0.119** (0.018) 235 Cross section30 Income from top 3 cash earning sources (in 1,000 Leones) 746.94 -21.773 (73.069) 236 Cross section 19
Structured Community Activity (SCA) Outcome: Mean for Treatment Standard N Controls Effect Error (1) (2) (3) (4)Panel A. Collective Action and the Building Materials VouchersGoBifo Mean Effect for SCA #1 (13 outcomes in total) 0.00 -0.057 (0.053) 236Proportion of communities that redeemed any vouchers at the buildingmaterial supply store 0.54 -0.01 (0.06) 236Average number of vouchers redeemed at the store (out of six) 2.95 0.11 (0.35) 236Proportion of communities that held a meeting after the research teamleft to discuss what to do with the vouchers 0.98 -0.05* (0.02) 231Panel C. Community Use of TarpaulinGoBifo Mean Effect for SCA #3 (18 outcomes in total) 0.00 -0.032 (0.045) 236Proportion of communities that held a meeting after the research teamleft to discuss what to do with the tarp 0.98 -0.03 (0.02) 233Proportion of communities that stored the tarp in a public place 0.06 0.06 (0.04) 225Proportion of communities that had used the tarp by the follow up visit(5 months after receipt) 0.90 -0.08+ (0.04) 233Given use of the tarp, proportion of communities that used the tarp in apublic way 0.86 0.02 (0.05) 161 20
Structured Community Activity (SCA) Outcome: Mean for Treatment Standard N Controls Effect Error (1) (2) (3) (4)Panel B. Participation in Gift Choice DeliberationGoBifo Mean Effect for SCA #2 (32 outcomes in total) 0.00 0.005 (0.036) 236Duration of gift choice deliberation (in minutes) 9.36 1.60 (1.13) 225Total adults in attendance at gift choice meeting 54.51 3.50 (3.20) 236Total women in attendance at gift choice meeting 24.99 1.99 (1.68) 236Total youths (approximately 18 to 35 years old) in attendance at giftchoice meeting 23.57 2.10 (1.38) 236Total number of public speakers during the deliberation 6.04 0.24 (0.40) 236Total number of women who spoke publicly during the deliberation 1.88 -0.19 (0.22) 236Total number of youths (approximately 18 to 35 years old) who spokepublicly 2.14 0.23 (0.24) 236Proportion of communities that held a vote during the deliberation 0.10 0.07 (0.04) 236 21
Illustrating the risk of “cherry-picking” (A)• Given our large number of outcome measures (318 in all), it is possible to selectively present one subset of outcomes for which CDD had a “positive” impact on institutions, and a second subset of outcomes that show the opposite impact.• Interpretation A: institutions deteriorated: – Heavy emphasis on participation led to “meeting fatigue” which translated into poor management and political disaffection and disengagement IGC, September 2011 Reshaping Institutions 22
Table 5: Alternative InterpretationsRow Survey question Mean for Treatment Standard N Hypothesis controls effect error (1) (2) (3) (4) (5) Panel A: Institutions Deteriorated 1 Did you attend any meeting to decide what to do with the tarp after our team left your community (not the original gift meeting)? 0.812 -0.037+ 0.021 236 H5 2 Everybody in the village had equal say in deciding how to use the 0.509 -0.106+ 0.058 232 H5 tarp 3 Correctly able to name what the tarp was used for or what the 0.589 -0.08+ 0.048 236 H9 communitys plan is for using the tarp 4 Has anyone in this community ever used the tarp? (verified by 0.897 -0.079+ 0.044 233 H4 supervisor physical assessment) 5 Supervisor asks to see the tarp at second round follow-up visit: can 0.836 -0.116* 0.051 232 H5 the community show you the tarp? 6 [Given not a member of the VDC] would you like to be a member 0.361 -0.043* 0.021 236 H10 of the VDC? 7 Is the current (or acting) village chief/Headman less than 35 years 0.044 -0.038+ 0.023 229 H12 old? 8 Did you vote in the local government election (2008)? 0.851 -0.036* 0.016 236 H10 IGC< September 2011 Reshaping Institutions 23
Illustrating the risk of “cherry-picking” (B)• Interpretation B: positive institutional spillovers from GoBifo – Experience with GoBifo encouraged communities to incorporate more democratic procedures into community decision making this created space for new leaders, and incited more engagement and activity including more involvement in womens groups and training of community teachers.• Illustrates the clear value of having a pre-analysis plan in place. IGC, September 2011 Reshaping Institutions 24
Table 5: Alternative InterpretationsRow Survey question Mean for Treatment Standard N Hypothesis controls effect error (1) (2) (3) (4) (5) Panel B: Institutions Improved 9 [Given community teachers at the school children in the 0.471 0.122+ 0.066 173 H4 community attend] were the community teachers ever trained?10 Are you a member of any womens groups (general)? 0.235 0.060** 0.021 236 H811 Did anyone take minutes (written record of what was said) at the 0.295 0.140* 0.063 227 H5 most recent community meeting?12 Enumerator record of whether a vote occurred during the gift 0.097 0.069+ 0.042 236 H5, H6 choice deliberation13 Respondent does not choose a chiefdom official or elder in response to "who had the most influence over how the tarpaulin is 0.543 0.058* 0.029 236 H6 used or whether to keep it in storage?"14 Correctly able to name the year of the next general elections 0.192 0.038* 0.018 236 H915 Respondent agrees with "Responsible young people can be good leaders" and not "Only older people are mature enough to be 0.762 0.038* 0.017 236 H6, H12 leaders"16 Proportion of female members of the VDC 0.209 0.066+ 0.037 151 H10 IGC, September 2011 Reshaping Institutions 25
Robustness checks• Were there threats to the research design? – Complete compliance with treatment group assignment – Baseline balance on observables across T/C groups• Did control communities benefit from spillovers? – GoBifo operated at the ward level as well, so targeting was possible. However, treatment households are if anything more likely to report direct benefits from ward project (15% versus 6%). – There are roughly an equal number of positive and negative coefficients on the POST indicator in our outcome equations.• Are our measures simply too blunt to detect subtle changes? – Large and diverse number of outcomes for each hypothesis, 318 in all. Consistent results across different data collection methods: HH surveys, direct observation, focus group discussions, and SCAs. IGC, September 2011 Reshaping Institutions 26
Conclusion: GoBifo results and questions• The CDD project was a reasonable mechanism for delivering local scale public goods in Sierra Leone. It gave women more voice temporarily yet did not lead to lasting changes in local collective action, village institutions, gender inclusion, or social norms.• Open questions for future research: – Did increased community participation during the project facilitate local public goods provision? In other words, would the positive Family A outcomes have materialized in the absence of activities designed to promote community involvement? – If not problems with coordination and participation, what is holding communities back from engaging in more collective action? – Would a project that explicitly attempted to weaken traditional chiefly authorities have had greater impact? IGC, September 2011 Reshaping Institutions 27
Our results in context• The comparative advantage of the World Bank and similar external donors may lie more in building development hardware than in instigating sustainable social change. Our results are positive on delivering concrete improvements in a challenging setting.• Setting up new organizations may be insufficient to promote social change since they can be co-opted by elites (Bardhan 2002; Gugerty and Kremer 2008), here, the chiefs.• Giving marginalized groups formal authority (i.e. political reservations for women in India, Beamen et al. 2009) may be more effective than indirect interventions like CDD that hope to shift informal institutions, especially when existing authorities are strong (chiefs in Sierra Leone).• As our results concern one program in one country, general implications are clearly speculative. More research is still needed to identify the precise interventions to reshape institutions to enhance collective action capacity while promoting accountability and inclusion. IGC, September 2011 Reshaping Institutions 28
END• Extra slides follow. U.C. Berkeley September 2011 Reshaping Institutions 29
U.C. Berkeley September 2011 Reshaping Institutions 30
Democratic Institutions and Collective Action Capacity:Results from a Field Experiment in Post-Conﬂict Liberia James Fearon, Macartan Humphreys, and Jeremy Weinstein Stanford and Columbia Universities September 20, 2011
Summary: The question. DFID, USAID, World Bank, and other major donors spend many billions each year on “Community Driven Development” and “Community Driven Reconstruction” projects. A common, core goal is to improve local governance by introducing elected development councils to select and manage projects. In various degrees, CDD/CDR projects also hope to improve material welfare. Does it work?
Summary: What we did. We worked with International Rescue Committee to randomly assign a CDR project (funded by DFID) to 42 of 83 communities in northern Liberia in Sept 2006. After program, measured impact on 1. material welfare and opinions about governance, etc, with surveys (March/April 2008), 2. a community-wide public goods game to assess impact on actual behavior.
Summary: Main ﬁndings (1). CDR had no apparent impact on material welfare. But we ﬁnd 1. Consistent evidence of CDR impact on attitudes expressed in follow-up survey. eg, support for dem process, inclusion of marginalized, levels of participation, trust in leaders. 2. CDR communities contributed signiﬁcantly more to raise funds for a small development project they selected (our public goods game). ⇒ CDR improved collective action capacity. Opposite results to Casey et al. Why? (speculative) No econ impact: IRC project forbid income-generating projects, was almost all small scale construction of buildings. GoBifo was mainly economic projects. coll action impact: We gave comm’s a decision problem perhaps closer to problems encountered in CDR program.
Summary: Main ﬁndings (2). Mechanisms. Why/how did this CDR program increase collective action capacity? (more speculative) No evidence that CDR made people more public spirited, caused more popular projects to be selected, improved comm sanctioning capacity, increased trust in NGOs. Evidence that CDR improved community leaders ability/inclination to mobilize community. (maybe) CDR program ⇒ greater leadership experience with similar organizational problems ⇒ learning by doing If so, then CDR worked via ↑ “leadership capital” more than by changing institutions, authority, and d-making practices. good, but less durable than institutional change?
Outline 1. A CDR program in Lofa County, Liberia. 2. Measurement strategies, including a public goods game. 3. Results on community contributions in game. 4. Mechanisms: Why did CDR have a positive impact? 5. Conclusions, policy implications.
The CDD/CDR program model CDR and CDD: Set up community-level institutions to democratically select and oversee development projects. Process (in our case): Initial sensitization of communities, formation of “advisory boards.” Election of Community Development Councils (CDCs). Participatory process to select quick impact ($1-2k) and larger development project (∼ $13k). CDCs oversee tendering of projects, implementation, and maintenance. Goals: Several, but mainly to improve community capacity for collection action in supplying or demanding public goods. Also, post-conﬂict reconciliation.
CDR implementation in Lofa County Program implemented by IRC in two districts of Lofa County, Liberia (Voinjama and Zorzor). Very hard hit by wars. Over 400 villages clustered into 83 similarly sized ‘communities’ for the purposes of CDR program administration. After baseline data collected, equally deserving/reachable villages randomly assigned to 42 treatment and 41 control groups in a public lottery attended by local chiefs and elders. CDR program runs October 2006 to March 2008. Community projects mainly emphasized small scale construction (latrines, renovation of town halls, schools, clinics, guesthouses).
Measurement strategy Panel survey of about 1600 households. Data collected before program (March/April 2006) and after (March/April 2008). Representative sample of Voinjama and Zorzor districts. Survey covered material welfare, political attitudes, collective action, political eﬃcacy, etc. Behavioral “real life” public goods game in 83 treatment and control communities (July-September 2008). Goal to observe collective action capacity and political practices six months after end of CDR program. Conducted in “hub” towns (largest village among the set of villages assigned to each community – avg about 140 households, or 800-1000 people).
Game protocol Community meeting at which community members told community could receive up to $420 to spend on a development project. Money received depends on: Community must complete form indicating how the money would be spent and which three people would handle the funds (“comm reps”). How much money a random sample of 24 people contributed to the project in a community-wide public goods game. One week later, team returns to village, collects form, samples 24 households, plays the public goods game, publicly counts the contributions, announces total, and provides the money to the three community reps.
Game protocol in more detail 83 communities participated in the game. In half, only women were selected to play the game; in the other half, household participants were equally divided between men and women (randomly assigned gender treatment). 24 randomly selected indivs played the game in each community, choosing how much of 300LD ($4.75) to contribute to the public good. Indiv decision made in private. 12 indivs had contributions multiplied by 2, other 12 by 5 (randomly assigned interest rate treatment). Surveys conducted with each indiv after s/he played. Additional surveys were completed with each of the three community representatives and the town chief.
Distribution of treatments CDR intervention total comm’s gender composition control treatment (participants) mixed groups (12m/12w) 20 22 42 (1008) women only (24w) 21 20 41 (984) total comm’s 41 (982) 42 (1008) 83 (1992) (participants) Notes: In all communities, 12 players were randomly assigned to have a high interest rate and 12 to low (equal m/w in mixed).
Advantages of behavioral game as msm’t strategy Measure CDR eﬀect on behavior in a cooperation problem rather than attitudes expressed in a survey. CDR could change survey responses w/o changing capacity or inclination to act. Can observe means by which communities select reps and decide on community project (“democratic process”). Was entirely disconnected from CDR program, implemented by an established Liberian NGO. Implementers generally did not know which communities were treatment and control. Not a lab experiment on indiv’s, but a real-world problem of collective action to raise money for community beneﬁt.
Results: Contribution levels 82 communities successfully completed the exercise (turned in a form, selected project, chose reps, had 24 households play game). Contribution rates very high overall in the public goods game Max communities could earn was 25,200LD: Average payout = 20,020 (80%), median = 20,850 (83%). Min was 10,900; max was 25,000. indiv contrib n % 15 300 1327 67.4 Frequency 10 200 201 10.2 100 231 11.7 5 0 209 10.6 0 0.4 0.5 0.6 0.7 0.8 0.9 1.0 payout/25200
Results: Main treatment eﬀects CDR program Gender comp. (allW) Interest Rate % max avg contrib % max avg contrib (5× vs 2×) Control 75.8 226 74.3 223 226 ATE 6.3 17 9.8 27 18 p-value .02 .03 .001 .003 0 Notes: CDR and gender comp treatment eﬀects are estimated at the community level, interest rate at indiv level. p values are two-tailed calculated by randomization inference for CDR and gender and with clustered se’s for interest rate. Avg contributions are in Liberian Dollars.
Heterogeneous treatment eﬀect Large CDR eﬀect in mixed gender groups, nothing in all W comm’s. Robust to conditioning on Quarter. % of max payout mixed all W no CDR 67.5 85.7 CDR 82.2 82.8 Community averages. Not explained by diﬀerent behavior of men and women: mixed all W women 221 248 (504) (960) men 226 (504) Avg contributions (n).
Possible mechanisms: Why did CDR work? 1. Direct eﬀects on 1.1 ‘Econ’ value for public good: eg, better project selection, or less graft by leaders. 1.2 ‘Social value’ for public good: eg, weight indiv’s put on community welfare vs cash. 1.3 Indiv costs/beneﬁts of contributing per se: eg, income eﬀect, more sanctioning, desire to please NGO, or democratic legitimacy eﬀect. 1.4 Coordination of expectations (if aspect of coordination game). 2. Or, indirect eﬀect on these via mobilization activities by community leaders. Maybe CDR increased experience with solving similar coll action problems.
Mechanisms evidence: Direct eﬀects. Effects of CDR Treatment on Intermediate Variables Satisfaction with Projects Trust in leaders No Anonymity Social Desirability Dem Procedure Expectations Accuracy ^ τ q q q q q q q All ^|Quarter τ q q q q q q q ^ τ q q q q q q q Mixed ^|Quarter τ q q q q q q q ^ τ q q q q q q q Women only ^ τ|Quarter q q q q q q q −0.4 −0.2 0 0.2 0.4 −0.4 −0.2 0 0.2 0.4 −0.4 −0.2 0 0.2 0.4 −0.4 −0.2 0 0.2 0.4 −0.4 −0.2 0 0.2 0.4 −0.4 −0.2 0 0.2 0.4 −0.4 −0.2 0 0.2 0.4 Economic Value Contribution Value Coordination Correlation between Intermediate Variables and Contributions (Non Experimental) Satisfaction with Projects Trust in leaders No Anonymity Social Desirability Dem Procedure Expectations Accuracy b q q q q q q q All b|CDR q q q q q q q b q q q q q q q Mixed b|CDR q q q q q q q b q q q q q q q Women only b|CDR q q q q q q q −100 −50 0 50 100 −100 −50 0 50 100 −100 −50 0 50 100 −100 −50 0 50 100 −100 −50 0 50 100 −100 −50 0 50 100 −100 −50 0 50 100 Economic Value Contribution Value Coordination
Conclusions: Material welfare Little or no evidence that CDR program in Liberia increased material welfare. Casey, Glennerster, Miguel ﬁnd material welfare impact of GoBifo CDR program in Sierra Leone, but no eﬀect on collective action capacity, d-mking processes. Why the diﬀerence? Possibly bec CDR in Liberia disallowed projects for income generation, whereas about 57% of GoBifo were this sort.
Conclusions: Collective action Clear evidence that in mixed comm’s, CDR caused greater mobilization eﬀorts which plausibly explain better performance in game. Suggests that CDR worked by mechanism of learning by doing: increasing human resources/experience via with related problems. Mobilization also occurs in all W, but not aﬀected by CDR treatment. Why? H: CDR program was deliberately mixed gender. Maybe comm’s used diﬀerent networks of people to mobilize for game, using women’s structures less aﬀected by CDR in all W. Mixed evidence for this: More “lady chiefs” are com rep’s in all W, but more (self reported) leaders of “women’s action groups” in mixed.
Conclusions: Policy implications. Possible policy implications (1): From the study/paper. CDR approach can work to increase community coll. action capacity. However, if mechanism is “leadership capital” rather than deeper institutional change, may not be large or strongly persistent. If CDD/CDR aid projects focus on setting up and working through gender-inclusive CDCs, but culture has tradition of gender-speciﬁc organization for coll action, may only help coll action in relations with outsiders.
Conclusions: Policy implications Possible policy implications (2): Some personal impressions. Aid programs are one-size-ﬁts-all templates. Some key assumptions of this CDR program wrong about this speciﬁc context. 1. Post-conﬂict reconciliation bet. Mandingo and Loma sorely needed, yes, but mainly inter- not intra-community/village. 2. Despite terrible war and displacement, social capital really high in these communities, and baseline attitudes about local democracy apparently strongly supportive. Big-donor RFP cycle and thematic lending ⇒ little incentives for NGOs to invest in location-speciﬁc knowledge, tailoring of programs to local context.
Community-Driven Development in Sierra Leone and Liberia Discussion at IGC Growth Week September 20, 2011 Eric Werker Harvard Business School
Random• Unintended and adverse consequences• Committing to hypotheses and publishing all results an amazing step
The dilemma of decentralization, liberally• The DRC is decentralized• But, in a post-conflict fragile state, the central government is by definition weak – projects little power or services into outlying areas• Are “youths” (18-35) disempowered? – Brutal civil war as social revolution
“Underdevelopment has become dangerous. This reinterpretation has become associated with a radicalization of development. Indeed, the incorporation of conflict resolution and societal reconstruction within aid policy – amounting to a commitment to transform societies as a whole – embodies this radicalization. Such a project, however, is beyond the capabilities or legitimacy of individual Northern governments.” (Mark Duffield, Global Governance and the New Wars)In Sierra Leone, the local institutions were part of a broader decentralizationoccurring under GoSL initiative, and created one extra level of administration.
Weak local institutions?• Local institutions seemed quite strong on their own; examine controls• Sierra Leone – 98% of communities held a meeting to decide what to do with the vouchers, tarp – 1 in 9 people speak to decide what to do with the gift, one third of whom women – 86% of villages used the tarp for public purposes• Liberia – Over 50% of respondents involved in at least three community organizations – Large majority has attended a community meeting in the last 6 months, and has spoke – 50% more community-focused in public goods games than typical lab experiments
Radically Ambitious• IRC : “This programme aims to empower local communities to be drivers and owners of their own recovery through community governance structures that stress choice and accountability…”• CGM : “created new structures designed to facilitate local development” (GoBifo) “…to increase social capital in grassroots communities… in order to promote more inclusive and effective development.” On the cheap? SL: $10/p/y for 3.5 years in capacity building, $16/p/y in grants Lib: $8/p/y for 2 years in grants, 1-3 projects
Do we get behavioral change from CDD?• SL: Money talks…• No, but we get institutional shells and stuff and income.• Fantastic – What’s the ERR of the investment?• LIB: Talking the talk, walking the walk (sometimes)• A marginal improvement when mixed gender games, plus survey response changes, but no stuff or income.“We like the GoBifo approach because it gives us confidence and helps us improve onother activities that are appropriate for human development... everybody in GoBifocommunities is always involved in planning and development. We manage and takeownership of the projects upon completion.” (chairman of a VDC)
Measuring up the tests• On the whole, extremely innovative and exciting, so the following will be unhelpful• Tested on pooling equilibria? – Lib: average 79% of payout goes to community – SL: • SCA3: 86% of villages used tarp in public way • SCA1,3: 98% of all communities discussed what to do – So what does treatment on the treated imply?• What is implied by better institutions? – SL SCA2: longer deliberation on whether to choose batteries or salt? More youth in attendance? Voting?
Measuring up the tests• On the whole, extremely innovative and exciting, so the following will be unhelpful• Tested on pooling equilibria? – Lib: average 79% of payout goes to community – SL: • SCA3: 86% of villages used tarp in public way • SCA1,3: 98% of all communities discussed what to do – So what does treatment on the treated imply?• What is implied by better institutions? – SL SCA2: longer deliberation on whether to choose batteries or salt? More youth in attendance? Voting?• Measuring social capital? – SL SCA1: not clear this requires more than a tiny coalition of elite
So what happened?• IRC Liberia – Community-based projects now work with existing local governance structures – M&E is straightforward baseline and indicator monitoring surveys
So what happened?• IRC Liberia – Community-based projects now work with existing local governance structures – M&E is straightforward baseline and indicator monitoring surveys• MPEA – Little information on work done 2006-08 because process of NGO reporting was not yet well established
So what happened?• IRC Liberia – Community-based projects now work with existing local governance structures – M&E is straightforward baseline and indicator monitoring surveys• MPEA – Little information on work done 2006-08 because process of NGO reporting was not yet well established• Multiple community development funds are being established – Inconsistent organizing principles, rocky start
Mismatch• In post-conflict situation, “building” institutions early and “working with” them later creates a mismatch – Govt not at the point, while its institutions are to be worked with, to have learned lessons immediately prior – Sierra Leone paper examining institutions with intent and commitment to permanently revive• This limits the potential impact, if the institutionalizing learning channel is cut off, to: – Temporary material relief, or social stability – Permanent material improvement, or shift in habits• Formal institutions could allow communities more practice with the processes, to actually build institutions, and enjoy the cultural shifts that might come with more practice.