Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Module 6
INSTRUMENTAL VARIABLES
REGRESSION DISCONTINUITY MODELS
PIPELINE METHODS
SHAHID KHANDKER
INTERNATIONAL FOOD POLICY...
1. Instrumental Variables
Why use IV?
Two sources of bias: program placement and
households self-selection; Examples: bias in formation
of income g...
IV Approach
Find a variable (or instrument, “Z”) highly correlated
with program placement or participation, but not
corre...
Model
Step 1: Regress T on instrument vector Z
(“First stage regression”):

Ti  Zi  Xi  ui
Step 2: Apply predicted ...
IV - Main Assumptions
(1) corr (Z , T) ≠ 0

Ti   Zi  Xi  ui
Yi (

Zi  

Xi  ui)  Xi i
(2) corr (Z, ) = ...
Sources of IVs: Randomization
• Savings program phased in randomly across
individuals
• Take-up not enforced across target...
Sources of IVs: Eligibility Rules
Grameen Bank in Bangladesh
(Pitt and Khandker, 1998):
• Program not allocated randomly
•...
• External shocks affecting program placement or
participation, but not affecting outcomes
Sources of IVs: External Shocks...
Interpretation of IV Estimate
Z does not perfectly capture participation/program
targeting
will only capture the program ...
Testing for Endogeneity
Best way: detailed information on program
implementation and participation
Examples: baseline su...
Testing for Endogeneity
 Can also do Wu-Hausman test
1. Isolate unobserved heterogeneity affecting T:
Regress T on Z and ...
Problems with IV: Weak Instruments
Corr (Z, ) ≠ 0

  Biased estimate of program effect
If more than one instrument ex...
Problems with IV: Weak Instruments
Larger standard errors, potentially inconsistent
estimates
Corr (Z ,T) is low
Yi   ...
IV with Panel Data

Yit Qit i vit, t 1,...,T,
Correlation between unobserved effect and
treatment variable T in a...
Conclusions
IV relaxes assumption of time-invariant heterogeneity
Instruments should be chosen carefully from detailed
d...
2. Regression Discontinuity
and Pipeline Methods
Approaches
Discontinuity design (RD): similar to IV - exogenous
eligibility rules used to compare participants and
nonpar...
Combining RD and Pipeline
Where there is a treatment is allocated based on some
exogenous criteria (RD), and potential pa...
Application - RD
Score = Si
Yi
(pre-intervention)
+
+
++
++
++
+
+
++
+
+
+
++
+
+
*
*
*
**
**
**
*
*
* *
*
*
*
**
*
**
**...
Specification Checks for RD
Graphing the predicted treatment effect
Plotting the density of the variable determining eli...
Pipeline Methods
Pipeline approach: can be randomized (e.g., PROGRESA)
or non-experimental (Jefes y Jefas Program in Arge...
RD and Pipeline Methods
Eligibility criteria combined with phased-in approach
Example: pension program awaiting budget e...
Overview
Advantage of RD: unbiased estimate of treatment effect
at the discontinuity
Potential problems:
• Interpretatio...
Case Studies:
IV AND REGRESSION DISCONTINUITY
How Do IV, RD and Pipeline Methods work?
IV exploits exogenous eligibility condition that affects only the
treatment but ...
Case Study 1:
Grameen Bank and Microcredit in
Bangladesh
Overview
Pitt and Khandker (1998) study the impact of group-
based microfinance programs in Bangladesh
Objective: to see...
Identifying Program Impacts
Relied on exogenous eligibility conditions based on
landholding (specifically, eligibility cu...
Data
Quasi-experimental dataset from 1991-92 of about
1,800 households across random sample of 29
thanas (87 villages)
A...
Data
 Household-level: eligible and ineligible households
from both types of villages
Thus, enough variation in targetin...
Instruments and Fixed Effects
• Instrument: land-based eligibility rule interacted
with household and village-level charac...
Initial Results
Using 1991 cross-section, Pitt and Khandker find that
when women are the program participants, program
cr...
Testing Sensitivity of Results
with Panel Data
Khandker (2005) uses the 1998-99 follow-up survey
to the 1991/92 survey to...
Case Study 2:
Impact of Social Protection in Argentina:
A Study Using Pipeline Method
Argentina Social Protection Program
Glasso and Ravallion (2004) examined the effect of a
large social protection program ...
Argentina Social Protection Program
Glasso and Ravallion exploited the program design to
construct the counterfactual; co...
Upcoming SlideShare
Loading in …5
×

Instrumental Variables, Regression Discontinuity Models, Pipeline Methods (Module 6)

125 views

Published on

The goal of this course is to provide policy analysts and project managers with the tools for evaluating the impact of a project, program or policy. This course provides information on the methods that can be used to measure the impact of a project, program or policy on the well-being of individuals and households. The course addresses the ways in which the results of an impact evaluation may be put to use – such as, to improve the design of projects and programs, as an input into cost-benefit analysis, and as a basis for policy decisions.

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Instrumental Variables, Regression Discontinuity Models, Pipeline Methods (Module 6)

  1. 1. Module 6 INSTRUMENTAL VARIABLES REGRESSION DISCONTINUITY MODELS PIPELINE METHODS SHAHID KHANDKER INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE (IFPRI)
  2. 2. 1. Instrumental Variables
  3. 3. Why use IV? Two sources of bias: program placement and households self-selection; Examples: bias in formation of income generating/credit networks; bias from targeting programs towards more entrepreneurial class or poor areas; such bias can also be time varying IV allows for unobserved bias in individual participation and / or program placement  Also accounts for errors in measuring participation as well as accounts for time varying endogeneity
  4. 4. IV Approach Find a variable (or instrument, “Z”) highly correlated with program placement or participation, but not correlated with unobserved characteristics affecting outcomes. Can have multiple instrumental variables for the treatment variable T
  5. 5. Model Step 1: Regress T on instrument vector Z (“First stage regression”):  Ti  Zi  Xi  ui Step 2: Apply predicted values in place of the endogenous variable in outcome equation Yi  (  Zi   Xi  ui ) Xi i   .IV program effect is
  6. 6. IV - Main Assumptions (1) corr (Z , T) ≠ 0  Ti   Zi  Xi  ui Yi (  Zi    Xi  ui)  Xi i (2) corr (Z, ) = 0: the exclusion restriction  
  7. 7. Sources of IVs: Randomization • Savings program phased in randomly across individuals • Take-up not enforced across targeted sample • Where unobserved bias affected take-up, targeting used as instrument Commitment Savings Product in Philippines(Karlan, Ashraf and Yin, 2006):
  8. 8. Sources of IVs: Eligibility Rules Grameen Bank in Bangladesh (Pitt and Khandker, 1998): • Program not allocated randomly • But program required that only households with 50 decimals of land could participate • Program participation is subject to gender-specific
  9. 9. • External shocks affecting program placement or participation, but not affecting outcomes Sources of IVs: External Shocks Rainfall and Civil Conflict in Africa (Miguel, Satyanath and Sergenti, 2004): • Examine impact of economic growth on civil conflict across 41 countries in Africa, 1998-99 • Use exogenous variation in rainfall to instrument for economic growth
  10. 10. Interpretation of IV Estimate Z does not perfectly capture participation/program targeting will only capture the program effect for a subset of participants    IV impact is therefore similar to ITT; also known as Local Average Treatment Effect (LATE)   
  11. 11. Testing for Endogeneity Best way: detailed information on program implementation and participation Examples: baseline survey, discussions with program officials (qualitative information helps as well) Can also help in designing instruments
  12. 12. Testing for Endogeneity  Can also do Wu-Hausman test 1. Isolate unobserved heterogeneity affecting T: Regress T on Z and the other exogenous X, and obtain the residuals. 2. Regress Y on X, Z, and residuals. If coefficient on residuals is significant, reject the null that T is exogenous.
  13. 13. Problems with IV: Weak Instruments Corr (Z, ) ≠ 0    Biased estimate of program effect If more than one instrument exists (equation is “overidentified,”) can test for this: 1. Estimate the structural equation by 2SLS, and obtain the residuals 2. Regress residuals on X and Z, obtain R2 and calculate test statistic q= (# instruments - # endogenous vars). nR2 ~ q 2 , 3. If reject corr (Z, ) = 0 .nR2  q 2 , 
  14. 14. Problems with IV: Weak Instruments Larger standard errors, potentially inconsistent estimates Corr (Z ,T) is low Yi   Ti i  cov(Yi,Zi)  cov[( Ti i),Zi]cov(Ti,Zi)  cov(Yi ,Zi ) cov(Ti ,Zi )  
  15. 15. IV with Panel Data  Yit Qit i vit, t 1,...,T, Correlation between unobserved effect and treatment variable T in addressed by differencing  i  Qit Instruments introduced to allow for correlation between some of the regressors in (such as T) and error itZ  Qit vit
  16. 16. Conclusions IV relaxes assumption of time-invariant heterogeneity Instruments should be chosen carefully from detailed data on program design and other exogenous factors affecting participation
  17. 17. 2. Regression Discontinuity and Pipeline Methods
  18. 18. Approaches Discontinuity design (RD): similar to IV - exogenous eligibility rules used to compare participants and nonparticipants Pipeline approach: exploit variation in the timing of program implementation, using as comparison group eligible units that have not yet received the program (e.g., PROGRESA) Examine outcomes for units just above and below cutoff point (so units are comparable)
  19. 19. Combining RD and Pipeline Where there is a treatment is allocated based on some exogenous criteria (RD), and potential participants (perhaps for a related program) are awaiting the intervention (Pipeline)
  20. 20. Application - RD Score = Si Yi (pre-intervention) + + ++ ++ ++ + + ++ + + + ++ + + * * * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * * * * * * * * * * * * * * * * * * * * * ** ** ** ** ** ** * * * * * * * * * ** ** ** ** * * * * * * * ** * Poor Non-poor + ++ ++ ++ + + ++ + + + ++ + ++ ++ ++ + + ++ + + + ++ + ++ ++ ++ + + ++ + + + ++ + ++ ++ ++ + + ++ + + + ++ + + + + + + + + + + + + + + + + + + + + + + ++ ++ ++ ++ ++ + + + + + + + + + + + ++ ++ ++ + + ++ + + + ++ + + + + s* Outcomes before intervention We need a variable S determining eligibility (such as age, asset holdings, etc.), with an eligibility cutoff of s*. Score = Si Yi (post-intervention) + + ++ ++ ++ + + ++ + + + ++ + * * * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * ** ** ** * * * * * * * ** * * * * * * * * * * * * * * * * * * * * * ** ** ** ** ** ** * * * * * * * * * ** ** ** ** * * * * * * * ** * Poor Non-poor ++ ++ ++ + + ++ + + + ++ + + + + ++ ++ ++ + + ++ + + + ++ + ++ ++ ++ + + ++ + + + ++ + ++ ++ ++ + + ++ + + + ++ + + + + + + + + + + + + + + + + + + + + + + ++ ++ ++ ++ ++ + + + + + + + + + + + ++ ++ ++ + + ++ + + + ++ + + + s* Outcomes after intervention Treatment effect
  21. 21. Specification Checks for RD Graphing the predicted treatment effect Plotting the density of the variable determining eligibility around the threshold Plotting the average values of the covariates around the threshold also can provide an indication of specification problems
  22. 22. Pipeline Methods Pipeline approach: can be randomized (e.g., PROGRESA) or non-experimental (Jefes y Jefas Program in Argentina) In non-experimental setting, can use matched double- difference approach as discussed in last lecture
  23. 23. RD and Pipeline Methods Eligibility criteria combined with phased-in approach Example: pension program awaiting budget expansion Example: local investment, such as road, is the source of additional market improvements, so that individuals around the road boundary would benefit from the future interventions Variation in potential exposure as a function of distance from road can be exploited as a source of identification
  24. 24. Overview Advantage of RD: unbiased estimate of treatment effect at the discontinuity Potential problems: • Interpretation, as with IV, is only for subset of relevant sample • Fewer observations to work with • Results are sensitive to functional form, including nonlinear relationships and interactions
  25. 25. Case Studies: IV AND REGRESSION DISCONTINUITY
  26. 26. How Do IV, RD and Pipeline Methods work? IV exploits exogenous eligibility condition that affects only the treatment but not the outcomes of interest; it allows for both time-invariant and time-varying heterogeneity causing sample selection bias; RD exploits exogenous eligibility rules to compare participants and non-participants around the cut-off point. Pipeline method constructs a comparison group from subjects who are eligible for the program but have not yet received it. Pitt and Khandker (1998) study the impact of group-based microfinance programs in Bangladesh
  27. 27. Case Study 1: Grameen Bank and Microcredit in Bangladesh
  28. 28. Overview Pitt and Khandker (1998) study the impact of group- based microfinance programs in Bangladesh Objective: to see if gender of participants matters for program impact on household per capita expenditure, school enrollment of children, household labor supply, assets Credit programs examined: BRAC, BRDB, Grameen Bank
  29. 29. Identifying Program Impacts Relied on exogenous eligibility conditions based on landholding (specifically, eligibility cut-off of maximum 1/2 acre of land owned) Also relied on fact that only men can participate in men’s credit groups, and likewise for women - easily enforceable, even where land criteria may not be Credit provision can therefore be measured by comparing returns to owning land
  30. 30. Data Quasi-experimental dataset from 1991-92 of about 1,800 households across random sample of 29 thanas (87 villages) About 1,540 targeted households from 24 thanas with at least one of the three programs; remaining 5 thanas had none Of targeted households, about 60% were participating in credit programs
  31. 31. Data  Household-level: eligible and ineligible households from both types of villages Thus, enough variation in targeting and eligibility to identify impacts:  Village-level: villages with and without program  Individual-level: program participants and nonparticipants across eligible households, in villages with program
  32. 32. Instruments and Fixed Effects • Instrument: land-based eligibility rule interacted with household and village-level characteristics • Gender constraint on groups can also identify program impacts across gender • Use village fixed-effects (for example, to account for why some villages have just men-only groups, and other villages have only female groups) Household-level endogeneity Village-level endogeneity
  33. 33. Initial Results Using 1991 cross-section, Pitt and Khandker find that when women are the program participants, program credit has a larger impact on household outcomes However, land criteria as an instrument is not strictly enforceable, so instrument efficiency was reduced but tested and found not too inefficient  Increase in annual household expenditure of 18 taka, compared to 11 taka for men  Average poverty reduction of about 5 percentage points
  34. 34. Testing Sensitivity of Results with Panel Data Khandker (2005) uses the 1998-99 follow-up survey to the 1991/92 survey to assess sensitivity of the earlier findings Identification strategy same as before, only that participation is now a dynamic variable  Average returns to female borrowing are about 21% in 1998/99  Average poverty reduction lower (about 2%): perhaps diminishing returns to borrowing
  35. 35. Case Study 2: Impact of Social Protection in Argentina: A Study Using Pipeline Method
  36. 36. Argentina Social Protection Program Glasso and Ravallion (2004) examined the effect of a large social protection program called, Jefes Y Jefas, in Argentina introduced during the 2001 economic crisis. Public safety net provided income to families with dependents for whom main source of earnings lost. However, not all the applicants received government support around the same time; there was a waiting period, which provides a case for applying pipeline method
  37. 37. Argentina Social Protection Program Glasso and Ravallion exploited the program design to construct the counterfactual; comparison units were constructed from a subset of applicants who had not yet received the program Participants were matched to comparison observations on the basis of propensity score matching method; Differences in outcomes between these groups provides an estimate of program effect However, the data revealed that the program’s eligibility criteria was not strictly enforced and that 80% of eligibles did not receive the program

×