SDVC Project Evaluation Design_Establishing Two Counterfactuals

1,022 views

Published on

This presentation from the International Food Policy Research Institute (IFPRI) provides an overview of the CARE Strengthening the Dairy Value Chain Project impact evaluation design.

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,022
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

SDVC Project Evaluation Design_Establishing Two Counterfactuals

  1. 1. Evaluating the Dairy Value Chain Project in Bangladesh: Baseline Study Akhter Ahmed International Food Policy Research Institute Seminar at CARE-Bangladesh Dhaka May 28, 2009
  2. 2. Storyline SDVCP objectives Evaluation methodology Baseline data Characteristics of survey households Gender related issues Dairy farming practices Qualitative review of value chain actorsINTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
  3. 3. SDVCP objectives1. Improve milk collection systems in rural and remote areas2. Improve access to inputs, markets, and services by mobilizing groups of poor farmers, producers, and char dwellers3. Improve the milk transport network4. Ensure access to quality service at the producer level5. Improve the policy environmentINTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
  4. 4. The baseline study CARE has commissioned IFPRI to conduct a thorough evaluation of the SDVCP that would allow CARE to make informed decision of whether to close, revise, extend, or expand the SDVCP CARE has also contracted the Data Analysis and Technical Assistance Limited (DATA) to collect quantitative and qualitative information for the evaluation, under guidance and supervision of IFPRI This baseline study is a part of the evaluation. It was specifically designed to permit a scientific and rigorous evaluation of impacts of the SDVCP through follow-up studiesINTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
  5. 5. Impact Evaluation Methodology Impact is the difference between outcomes (e.g., income, school enrollment, women’s empowerment, etc) with the program and without it The goal of impact evaluation is to measure the this difference in a way that can attribute the difference to the program, and only the program Use Difference-in-differences method that compares observed changes in the outcomes for program participants (treatment) and non-participating comparison group (control), before and after the program Combine with propensity score matching to adjust for pre- program differencesINTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
  6. 6. Propensity Score Matching (PSM) Match program participants (treatment) with non- participants (control) at baseline (before intervention) Each program participant will be paired with a non-participant that is similar Use PSM to pick an ideal comparison group from the baseline survey data The comparison group will be matched to the treatment group using “propensity score” Propensity score is predicted probability of participation given observed pre-program characteristics of participants and non- participantsINTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
  7. 7. Illustrating Difference-in-Difference Estimate of Average Program Effect PA   Impact = (PA - CA) - (PB - CB) Program CA Control PB=CB Baseline Follow-up (Before) (After)INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
  8. 8. Constructing the counterfactual: Two control groups The control or comparison groups are comprised of eligible but non-participant households Two control or comparison groups of households have been created to assess the impact and to capture the potential spillover effects The nature of SDVCP interventions may generate spillover effect of the project. For example, if new dairy production technologies are introduced, non-beneficiaries may copy these Control 1 households have been selected from unions where the SDVCP is operating Control 2 households have been selected from upazilas without any milk chilling pants in the nine project districtsINTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
  9. 9. Baseline data Used both quantitative and qualitative data for the baseline study Quantitative data came from a comprehensive household survey designed by IFPRI and carried out by DATA The survey questionnaire was designed to collect information on multiple topics, including household demographic composition, level of education, school participation, occupation and employment, dwelling characteristics, assets, food and nonfood expenditures, morbidity, economic shocks, anthropometric measurements of children and women, and a detail module on dairy farming practicesINTERNATIONAL FOOD POLICY RESEARCH INSTITUTE
  10. 10. Baseline data … The household survey was carried out in 9 SDVCP districts, 27 upazilas, and 60 villages Sample size: Determined by power calculation with design effect. Total sample of 1,510 households, of which 659 program participants, 425 Control 1, and 426 Control 2 households Survey started on August 20, 2008, and completed on September 14, 2008. Data entry completed by end October. Data cleaning, including logical consistency checking and data validation completed by mid-January 2009 Qualitative data collection: Used key-informant interview, focus group discussion, and observation methodsINTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

×