Improving market access of farmer groups in Uganda: evaluating the role of working capital
Upcoming SlideShare
Loading in...5
×
 

Improving market access of farmer groups in Uganda: evaluating the role of working capital

on

  • 10,116 views

 

Statistics

Views

Total Views
10,116
Views on SlideShare
8,802
Embed Views
1,314

Actions

Likes
0
Downloads
5
Comments
0

3 Embeds 1,314

http://unjobs.org 1311
http://www.unjobs.org 2
http://translate.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Improving market access of farmer groups in Uganda: evaluating the role of working capital Improving market access of farmer groups in Uganda: evaluating the role of working capital Presentation Transcript

  • Impact evaluation An application to farmers groups in Uganda Improving market access of farmer groups in Uganda: evaluating the role of working capital Ruth Vargas Hill and Eduardo Maruyama May 9, 2012
  • Impact evaluation An application to farmers groups in Uganda Outline Impact evaluation Introduction RCTs An application to farmers groups in Uganda Introduction Implementation Results Concluding remarks
  • Impact evaluation An application to farmers groups in Uganda Why evaluate? • Evaluating interventions (policies or programs) helps: • Understand the actual rather than the anticipated effects of programs. • Determine how to design new programs. • Determine the most cost-effective approach to achieve a desired goal.
  • Impact evaluation An application to farmers groups in Uganda Estimating impact: introduction • When we conduct impact evaluation we assess how a program affects the well-being or welfare of individuals, households or communities: • Profitability of agricultural production • Increased income or consumption (or other measures of welfare) of rural households • Poverty levels or growth rates at the community level
  • Impact evaluation An application to farmers groups in Uganda Impact evaluation versus other M&E tools • Impact evaluation is different from other M&E tools in that it focuses on discerning the impact of the program from all other confounding effects. • The focus of impact evaluation is providing evidence of the causal link between an intervention and an outcome. • This is why impact evaluation is a powerful too, but also what makes it difficult to implement in practice.
  • Impact evaluation An application to farmers groups in Uganda Impact evaluation versus other M&E tools DIFFICULTY OF  Low High SHOWING CAUSALITY Inputs Outputs Outcomes Impacts Example: A program of providing advice on a new technology to farmers Visits by  Increased  extensions  Knowledge of  Use of the  yields, higher  agents,  the new  new  farm profits,  physical inputs  technology technology improved  (such as  consumption seeds)
  • Impact evaluation An application to farmers groups in Uganda Essential component: counterfactual • Difficulty is determining what would have happened to the individuals or communities of interest in absence of the project. • We are interested in the difference in an outcome for an individual with and without the intervention. • Problem: can only observe people in one state of the world at one time • The key component to an impact evaluation is to construct a suitable comparison group to proxy for the counterfactual.
  • Impact evaluation An application to farmers groups in Uganda Before and after comparisons • Why not collect data on individuals before and after intervention (the Reflexive)? Difference in income, etc, would be due to project • Problem: many things change over time, including the project • The country is growing and profits are rising. Is this due to the program or would have occurred in absence of program? • This is particularly a problem for agricultural interventions: many factors affect yield (weather, availability of inputs) and prices in a given year.
  • Impact evaluation An application to farmers groups in Uganda Comparison groups • Instead of using before/after comparisons, we need to use comparison groups to proxy for the counterfactual • Two core problems in finding suitable groups: • Programs are targeted • Recipients receive intervention for particular reason • Participation is voluntary • Individuals who participate differ in observable and unobservable ways (selection bias) • Hence, a comparison of participants and an arbitrary group of non-participants can lead to misleading or incorrect results
  • Impact evaluation An application to farmers groups in Uganda Randomizing to create a true comparison group • We need a comparison group that is as identical in observable and unobservable dimensions as possible, to those receiving the program, and a comparison group that will not receive spillover benefits. • Number of techniques: • Randomized control trials (the gold standard) • Careful matching techniques: IV, propensity score matching, regression discontinuity design
  • Impact evaluation An application to farmers groups in Uganda Randomized Control Trials (RCTs) • In RCTs, participation in a policy (or usually eligibility to participate in a policy) is randomly assigned. • This is done to ensure that the only difference between those in and out of an intervention, is their participation, and as a result any difference between participants and non-participants can be attributed to the program alone. • Because participation (treatment) is randomized, the non-treatment outcomes between those that are not treated and those that are treated is equal.
  • Impact evaluation An application to farmers groups in Uganda Households or groups of households
  • Impact evaluation An application to farmers groups in Uganda Households or groups of households C T C C C T T C C T C T C T C T C T C C C C T T T C T
  • Impact evaluation An application to farmers groups in Uganda Does randomization create a true comparison group? • We can test that they are equal by collecting data on the two groups before the intervention and checking that the average characteristics of the two groups are the same. • For the treatment and control groups to be statistically equal you need a large number of each. Cannot have one treated household and one control household. • Means that you cannot use this method to answer questions about country policy changes (e.g. fiscal policy changes). • There are stragegies that can be used to ensure that the treatment and control groups are equal (e.g. stratification).
  • Impact evaluation An application to farmers groups in Uganda How do we estimate impact by randomizing? • Identify the outcome we are interested in (e.g. yields, amount of output marketed, price received) • Estimate the average of the outcome in the treatment group. • Estimate the average of the outcome in the control group. • Calculate the difference of these averages and test to see if the two averages are significantly different from each other. • Average Treatment Effect • Note: it is just differences in the AVERAGE outcome that are estimated.
  • Impact evaluation An application to farmers groups in Uganda Challenges to estimating impact • Sometimes the effect of the program is small. • Or there are many other factors affecting the outcome of interest that it is hard to see if a difference is statistically different between two groups. • We try and control for this in two ways: • Include a large number of households in treatment and control. This increases our power to detect a small effect. • Collect data on characteristics of the household that may influence the outcome variable at baseline (including the pre-intervention outcome of interest)
  • Impact evaluation An application to farmers groups in Uganda Other challenges to estimating impact • Are we sure that the intervention had no impact on the control group? Are there no spillover effects? (E.g. on prices) • Was there any attrition as a result of the program that means we miss capturing some of the impact? For example did people migrate as a result of the program? If so, we will miss capturing the effect of the program on these people. • We randomized to avoid selection bias, but some of it still may remain: • Did everyone in the treatment group participate as expected? • Did anyone in the control group participate even if they were not meant to?
  • Impact evaluation An application to farmers groups in Uganda Selection bias Not in evaluation Target Population Treatment Participants group Evaluation Random No-Shows Sample Assignment Non- Control group Participants Cross-overs 36
  • Impact evaluation An application to farmers groups in Uganda Handling selection bias • Intent to treat (ITT): • Average impact of program in practice: treats all noncompliars as treated, and treats all crossovers as remaining in the control • Problem: power is reduced by noncompliance and does not provide an idea of what the average impact of the program on the treated is. • Treatment on the treated (ToT): • Instruments for take-up with assignment: gives an idea of the average impact of the program for a specific group
  • Impact evaluation An application to farmers groups in Uganda Summary of advantages and disadvantages • Powerful method to identify causal impact of a policy or program. • Careful design is needed to ensure you are able to detect changes • Can be expensive: baseline and follow-up, a large number of participants (especially if it is to be representative) • Only valid in some circumstances: randomization over a number of units needs to be possible. • Provides information on the average outcome. • Internally valid, repetition and a theory of change needed to make predictions from results (external validity).
  • Impact evaluation An application to farmers groups in Uganda External validity • Tells us whether something worked in a specific context, understanding whether an intervention would work again in a different setting, external validity, is very difficult to know. • Great benefit of few assumptions, comes with great cost ”narrowness of scope” (Cartwright 2007). • Any external validity involves some assumptions about the project working in different conditions. • Yet replicating a project is almost impossible, replicating triggers of mechanisms that produce the change is often more possible. • Result: we need to repeat impact evaluations and also have a theory of change to know what will work in the future, i.e. to really learn. • But better than an approach that is not internally valid. Predictions cannot be made from this either.
  • Impact evaluation An application to farmers groups in Uganda Other concerns in learning from results • General equilibrium effects. • Corruption in implementing a large scale. • Capacity to implement at a large scale. • Overlap between new environment and old (example of medicine) • Not automatic to go from experiments to learning and policy advice.
  • Impact evaluation An application to farmers groups in Uganda Another approach • We need a theory of change that guides us in going from one-off impact evaluation to general lessons. This means make assumptions and, ideally, use experiments to test and refine these assumptions. • When designing impact evaluation for this purpose, it often looks quite different: • Theory of change influences the design of the impact evaluation. • Often identifying the differential impact of different treatments, rather than the impact of one treatment against baseline.
  • Impact evaluation An application to farmers groups in Uganda Outline Impact evaluation Introduction RCTs An application to farmers groups in Uganda Introduction Implementation Results Concluding remarks
  • Impact evaluation An application to farmers groups in Uganda Introduction • Smallholder agriculture in Sub-Saharan Africa is largely exposed to pervasive market failures, translating into missed opportunities and sub-optimal economic behavior. • These failures are often rooted in the importance of economies of scale in procuring inputs and marketing produce. • By engaging in markets collectively through a farmers group, smallholders can overcome economies of scale. • Despite the renewed interest from governments and donor agencies in farmers groups as a means to overcome these market failures, evidence shows that they have so far had limited success.
  • Impact evaluation An application to farmers groups in Uganda Ugandan context • The majority of Ugandan farmers sell their (unprocessed) produce at harvest time to itinerant traders at the farm-gate. • Survey of farmers groups engaged in some form of output marketing revealed that: • Farmers get a higher price when they sell collectively. • Yet few farmers sell through the marketing group of which they are a member (only 47% make sales through group) • Farmers are less likely to sell collectively when they are liquidity constrained and in need of emergency money. • Groups that offer cash on delivery of produce (rather than payment some days later) have a higher proportion of members selling through the group.
  • Impact evaluation An application to farmers groups in Uganda Key impact question • Would providing working capital loans to farmers groups so that they can provide cash on delivery, improve marketing outcomes for farmers? • We cannot infer this from the baseline data: good groups may be better at collective sales and better able to access finance which allows payment on delivery. • We would like to compare groups of similar quality and see if working capital loans increase sales amongst those that received them.
  • Impact evaluation An application to farmers groups in Uganda Testing a theory of change • Farmer groups can offer higher prices but because of the waiting times involved in receiving payment, farmers find it costly to sell though the group. • Farmers are liquidity constrained and often sell coffee to meet urgent financial needs, so even small delays in payment can be problematic. • Waiting for payment involves a high level of trust in the ability of the group to market and transparency. There is a risk if the groups cannot be trusted. • Enabling groups to make payment on delivery through a working capital loan will reduce the cost of selling through the group. • More farmers will sell through the group and receive higher prices as a result.
  • Impact evaluation An application to farmers groups in Uganda The impact of working capital credit • Randomized provision of working capital credit to farmers groups that had already been engaged in output marketing: • Provide selected groups with a fund to make partial cash payments to farmers upon delivery of produce. Once the group makes a sale the fund is replenished and farmers are given the remaining balance. “Cash on Delivery” (CoD) • Assess the impact of this credit on the proportion of produce sold through the group and on the price farmers received. • Understand why this worked? • Did this work for farmers likely to face liquidity constraints, or only in groups where trust was already high? • Implement an intervention on improved transparency to randomly selected groups to improve trust in some groups. Is the working capital intervention just as effective in those groups with the transparency intervention? • Information on Sales (IoS): SMS system to provide members with specific information about transactions made by the group (final sale price, fees deducted, etc.), plus reinforced training on book-keeping.
  • Impact evaluation An application to farmers groups in Uganda Coffee/maize group marketing structure • Farmers groups (“PO”s for producer organizations) are typically grouped under associations (DCs for district committees). • The PO handles bulking and coordination of transport with members at the village level. • The DC take care of collection and in some cases value addition to the next stage of marketing. • In most cases, a service organization offers support to DCs and POs through lobbying, access to extension and additional marketing services.
  • Impact evaluation An application to farmers groups in Uganda Implementation • The study was carried out in 9 DCs marketing coffee and maize, containing 165 POs under them. • March 2010, Baseline survey: • A 3-tiered survey which collected detailed information on DCs, POs, and member households. • Full roster of members for each PO, and a complete household survey for at least 2 members of each group. • November 2010 September 2011, implemented intervention in randomly selected groups: • Provided working capital credit to randomly selected POs. • Provided SMS information on deliveries to randomly selected POs. • October 2011, Follow-up survey. • Collected detailed information on POs and member households. • Collected administrative data from the DC records to obtain more reliable delivery data.
  • Impact evaluation An application to farmers groups in Uganda Randomization strategy • We randomized the interventions at the PO level stratifying the sample by DC, since the sample size is not large enough at the DC level and the risk of spill-overs is too high at the household level. • POs in each DC are randomly assigned into 4 groups: (1) CoD, (2) IoS, (3) CoD + IoS, and (4) none. • The fund for the CoD was managed by the DC, and vouchers were given to treated POs so their members could request immediate partial payments for output deliveries. • For the IoS intervention, a DC staff member was selected to send the messages to key farmers in the treated POs.
  • Impact evaluation An application to farmers groups in Uganda Are control and treatment groups equal? Control CoD IoS Both (mean) Members 24.256 2.194 4.558 1.597 (3.106)∗∗∗ (4.365) (4.289) (4.339) Years since foundation 4.400 1.039 0.460 1.014 (0.526)∗∗∗ (0.739) (0.731) (0.739) Marketing services 0.825 -0.020 -0.081 -0.093 (0.066)∗∗∗ (0.093) (0.092) (0.093) Output bulked (kgs.) 854.025 -240.708 -192.862 -325.440 (236.818)∗∗∗ (332.863) (329.018) (332.863) Female leader 0.250 -0.006 -0.064 -0.030 (0.067)∗∗∗ (0.094) (0.093) (0.094) Leader’s age 52.200 -4.639 -2.153 0.190 (1.874)∗∗∗ (2.634)∗ (2.604) (2.634) Leader’s schooling 8.025 0.073 0.208 -0.562 (0.460)∗∗∗ (0.647) (0.639) (0.647) POs 40 41 43 41
  • Impact evaluation An application to farmers groups in Uganda • Implementing the interventions represented a major challenge: 1. The POs in our study are spread over many regions in the country. 2. Implementation needed to be done by a 3rd party, to avoid service organizations and DCs contaminating the PO-level randomization strategy. 3. In order to avoid undesired heterogeneity in implementation, training, and monitoring of the interventions, a single implementing agency was favored over several regional organizations. 4. Training and distribution of vouchers within the PO was delegated to PO leaders in some DCs. • Our own monitoring activities as well as the follow-up survey indicate implementation was problematic. • Some cross-over and no-shows for CoD intervention • Overall implementation of IoS intervention.
  • Impact evaluation An application to farmers groups in Uganda Empirical strategy McKenzie (2011) shows that using baseline data on the outcome variable of interest, allows more power to detect impact. Therefore, for our analysis we estimate: Yi,1 = α + γj Di,j + θYi,0 + εi,1 j
  • Impact evaluation An application to farmers groups in Uganda Results Table 2.1: Impact of interventions on produce deliveries PO Household Kgs. P(Delivery) Kgs. CoD only 747.826 0.186 162.700 (325.294)∗∗ (0.079)∗∗ (88.400)∗ IoS only 355.764 0.089 62.660 (320.386) (0.078) (87.060) Both -584.566 0.101 122.000 (455.073) (0.077) (86.280) Observations 165 244 243 R2 0.422 0.269 0.084
  • Impact evaluation An application to farmers groups in Uganda Results Table 2.2: Impact of selling through PO on transaction features Price Days between sale and payment Sold through PO 0.858 -6.540 (instrumented) (0.477)∗ (21.170) Observations 193 192 R2 0.704 0.210
  • Impact evaluation An application to farmers groups in Uganda Concluding remarks • Despite implementation problems, the CoD intervention has a significant impact on group marketing. • CoD increases the probability a household will sell through the group, how much each household will sell, and the total amount sold by the group. • By encouraging farmers to sell through the group, CoD has an effect on increasing the price they receive. •