Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Differences-in-Differences

213 views

Published on

2016. 7. 27 Presentation (Co-presenter: Jia Li)
Research Method for Political Science III (Instructor: Yuki Yanai)
Graduate School of Law, Kobe University, Japan

Some characters in this slide are invisible. High-resolution slide is available on my homepage (http://www.jaysong.net).

Published in: Data & Analytics
  • Be the first to comment

Differences-in-Differences

  1. 1. Research Method for Political Science III Di↵erences-in-Di↵erences Jia Li Jaehyun Song Kobe University 2016-07-27 Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 0 / 38
  2. 2. Table of Contents 1 Review 2 Application Fouirnaies and Mutlu-Eren 2015 3 Practice Background Graphical Explanation Estimating Causal E↵ects Using Linear Regression 4 Standard Errors in Di↵-in-Di↵ Estimation 5 Synthetic Control Method Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 1 / 38
  3. 3. Review Review of DID When do we usually use DID estimation? The treatment and control groups di↵er systematically e.g. For job training program, if workers who took the training are predominantly uneducated, we may find an average earnings of treatment group is lower than that of the control group. Panel or repeated cross sectional data before and after the experiment(e.g. program, policy) are available The common trends assumption is satisfied Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 2 / 38
  4. 4. Application Fouirnaies and Mutlu-Eren 2015 English Bacon: Research Question Research Question Do government parties allocate more resources to local councils that are controlled by their own party? Copartisanit = ( 1 if majorityit 2 Gt 0 otherwise i: Local council(2 (1, 2, . . . , 466)) t: Year(1992⇠2012) G: Government party Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 3 / 38
  5. 5. Application Fouirnaies and Mutlu-Eren 2015 English Bacon: Comparing the Two Groups We are interested in comparing the Specific Grant(SG) allocated to the local councils(Copartisanit = 1) and the others. Identification Strategy E[SGit|Copartisanit = 1] E[SGit|Copartisanit = 0] Omitted Variable Bias Economic growth ) Specific Grant # ) More votes to the prime minister’s party Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 4 / 38
  6. 6. Application Fouirnaies and Mutlu-Eren 2015 English Bacon: Identification Strategy Identification Strategy ySG i,t+k = 1Copartisanit + ↵i + t + ↵it + Xit + "i,t+k ySG i,t+k: SG per capital allocated to i at t + k(logged) ↵i: Fixed e↵ect (local councils) t: Fixed e↵ect (time) Xit: Control variables Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 5 / 38
  7. 7. Application Fouirnaies and Mutlu-Eren 2015 English Bacon: Common-Trends Assumption Including council-specific trends variables(↵it) can mitigate the Common-Trends Assumption, but the assumption can still be violated because of nonlinear trends. New Identification Strategy(Relaxing the Assumption) Di↵erences-in-Di↵erences-in-Di↵erences Estimator ySG i,t+k yFG i,t+k = 1Copartisanit + ↵i + t + ↵it + Xit + "i,t+k yFG i,t+k Formula Grant per capital allocated to i at t + k(logged) Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 6 / 38
  8. 8. Application Fouirnaies and Mutlu-Eren 2015 English Bacon: Other Models Case 1: Is the e↵ect larger before elections? yi,t+k = ↵i + t + ↵it + 1Copartisanit + 2ElectYeari,t+k + 3(Copartisanit ⇥ ElectYeari,t+k) + "i,t+k ElectYeari,t+k A dummy variable indicating whether there is a local election in i at t + k. Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 7 / 38
  9. 9. Application Fouirnaies and Mutlu-Eren 2015 English Bacon: Other Models Case 2: How do goverments strategically manipulate the timing of grant allocation? yi,t+k = ↵i + t + ↵it + 1Copartisanit + 2YearToElecti,t+k + 3(Copartisanit ⇥ YearToElecti,t+k) + "i,t+k YearToElecti,t+k A variable counting the number of years to the next local election in i at t + k. Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 7 / 38
  10. 10. Application Fouirnaies and Mutlu-Eren 2015 English Bacon: Other Models Case 3: Is the e↵ect strongest in councils that provide citizen-focused services and hold relatively infrequent elections? yi,t+k = ↵i + t + ↵it + 1Copartisanit + 2(Copartisasnit ⇥ InfrequentElectionsi) + 3(Copartisanit ⇥ UpperTieri) + 4(Copartisanit ⇥ UpperTieri ⇥ InfrequentElectionsi) +"i,t+k InfrequentElectionsi A dummy variable indicating whether i holds elections only once every four years or more often. UpperTieri A dummy variable indicating whether the i refers to a top-tier council Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 7 / 38
  11. 11. Application Fouirnaies and Mutlu-Eren 2015 English Bacon: Other Models Case 4: Is the e↵ect stronger in “swing” councils? yi,t+k = ↵i + t + ↵it + 1Copartisanit + 2Swingit + 3(Copartisanit ⇥ Swingit) + "i,t+k Swingit A dummy variable that takes the value 1 if neither the government nor the opposition held an absolute majority of the seats in i before election t. Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 7 / 38
  12. 12. Practice Background e-Vote in Kyoto Vote using touch panel devices NOT PC or cell phone Some wards in Kyoto(city) adopted e-Vote in 2004(Higashiyama ward) and 2008(Kamigyo ward) Kyoto city has eleven wards. (Unfortunately, the wards abolish the e-Voting.) Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 8 / 38
  13. 13. Practice Background e-Vote in Kyoto Figure: e-Vote Device Source: http://blogimg.goo.ne.jp/user_image/70/fc/e198dc314f386001a5c789d5d18fa059.jpg Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 8 / 38
  14. 14. Practice Background e-Vote in Kyoto Source: Wikipedia Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 8 / 38
  15. 15. Practice Background Does e-Vote Make Democracy Great Again? 1 e-Vote may reduce voting costs(. . . ?) 2 e-Vote may reduce mistakes in filling ballots. H1 e-Vote makes voters turnout higher. H2 e-Vote makes spoilt votes reduce. We can estimate its causal e↵ects using Di↵-in-Di↵. Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 9 / 38
  16. 16. Practice Graphical Explanation Graphical Explanation: H1 E↵ect size: 0.0521 0.300.350.400.450.50 Year VoterTurnout 2000 2004 Higashiyama Fushimi Counterfactual Higashiyama Does it meet “the parallel assumption”? Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 10 / 38
  17. 17. Practice Graphical Explanation Graphical Explanation: H1 Collect data from other wards 0.300.350.400.450.50 Year VoterTurnout 2000 2004 Wards except Higashiyama Mean of the others Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 10 / 38
  18. 18. Practice Graphical Explanation Graphical Explanation: H1 E↵ect size: 0.05345 0.300.350.400.450.50 Year VoterTurnout 2000 2004 Higashiyama The others mean of the others Counterfactual Higashiyama Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 10 / 38
  19. 19. Practice Graphical Explanation Graphical Explanation: H2 E↵ect size: -0.01538832 0.0000.0050.0100.0150.0200.0250.030 Year SpoiltVotes 2000 2004 Higashiyama Fushimi Counterfactual Higashiyama Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 11 / 38
  20. 20. Practice Graphical Explanation Graphical Explanation: H2 Check the parallel assumption 0.0000.0050.0100.0150.0200.0250.030 Year SpoiltVotes 2000 2004 Wards except Higashiyama Mean of the others Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 11 / 38
  21. 21. Practice Graphical Explanation Graphical Explanation: H2 E↵ect size: -0.01647571 0.0000.0050.0100.0150.0200.0250.030 Year SpoiltVotes 2000 2004 Higashiyama The others mean of the others Counterfactual Higashiyama Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 11 / 38
  22. 22. Practice Estimating Causal E↵ects Using Linear Regression Prepare Let’s Practice! Please launch R, and load a package and the dataset library(dplyr) # Thanks, Hadley! dfURL <- "http://jaysong.net/RMPS3/eVoteKyoto.csv" DD_df <- read.csv(dfURL) Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 12 / 38
  23. 23. Practice Estimating Causal E↵ects Using Linear Regression Data Structure ID ID Ward J Ward name(Japanese) Ward E Ward name(English) year Year(2000⇠2016) trend Trend Indicator(1⇠4) eVote Treatment Variable(e-Vote) turnout Voter turnout spoilt Spoilt votes Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 13 / 38
  24. 24. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 1: Comparing the Two Points Turnoutwt = ↵ + eVotewt + FushimiX j=Kamigyo jWardj + Year2004 df1 <- DD_df %>% filter(year <= 2004) H1Model1 <- lm(turnout ~ eVote + as.factor(WardID) + as.factor(year), data = df1) summary(H1Model1) Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 14 / 38
  25. 25. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 1: Comparing the Two Points Estimate Std. Error t value Pr>|t| Intercept 0.483075 0.002754 175.393 < 2e-16 eVote 0.053450 0.005508 9.703 4.60e-06 is exactly same to the result of graphical explanation. Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 15 / 38
  26. 26. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 1: Comparing All the Points Of course, we can use all the data. Turnoutwt = ↵ + eVotewt + FushimiX j=Kamigyo jWardj + 2016X k=2004 kYeark H1Model2 <- lm(turnout ~ eVote + as.factor(WardID) + as.factor(year), data = DD_df) summary(H1Model2) ------------------------------ Estimate Std. Error t value Pr(>|t|) (Intercept) 0.485430 0.004691 103.490 < 2e-16 eVote 0.024672 0.006248 3.949 0.000319 Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 16 / 38
  27. 27. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 1: Considering trend e↵ect How about to consider trend e↵ect? Turnoutwt = ↵ + eVotewt + FushimiX j=Kamigyo jWardj + 2014X k=2004 kYeark + FushimiX j=Kamigyo j(Wardj ⇥ Trend(t)) H1Model3 <- lm(turnout ~ eVote + as.factor(WardID) + as.factor(year) + as.factor(WardID) * trend, data = DD_df) summary(H1Model3) Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 17 / 38
  28. 28. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 1: Considering trend e↵ect Estimate Std. Error t value Pr(>|t|) (Intercept) 0.4818506 0.0060225 80.008 < 2e-16 eVote 0.0240453 0.0054012 4.452 0.000116 Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 18 / 38
  29. 29. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 2: Comparing the Two Points Spoiltwt = ↵ + eVotewt + FushimiX j=Kamigyo jWardj + kYear2004 H2Model1 <- lm(spoilt ~ eVote + as.factor(WardID) + as.factor(year), data = df1) summary(H2Model1) Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 19 / 38
  30. 30. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 2: Comparing the Two Points Estimate Std. Error t value Pr>|t| Intercept 0.0126155 0.0005573 22.636 3.04e-09 eVote -0.0164757 0.0011146 -14.782 1.28e-07 is also exactly same to the result of graphical explanation. Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 20 / 38
  31. 31. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 2: Comparing with All the Points Of course, we can still use all the data. H2Model2 <- lm(spoilt ~ eVote + as.factor(WardID) + as.factor(year), data = DD_df) summary(H2Model2) ------------------------------ Estimate Std. Error t value Pr(>|t|) (Intercept) 1.255e-02 9.919e-04 12.654 2.19e-15 eVote -1.884e-02 1.321e-03 -14.257 < 2e-16 Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 21 / 38
  32. 32. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 2: Considering trend e↵ect How about to consider trend e↵ect? Spoiltwt = ↵ + eVotewt + FushimiX j=Kamigyo jWardj + 2014X k=2004 kYeark + FushimiX j=Kamigyo j(Wardj ⇥ Trend(t)) H2Model3 <- lm(spoilt ~ eVote + as.factor(WardID) + as.factor(year) + as.factor(WardID) * trend, data = DD_df) summary(H2Model3) Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 22 / 38
  33. 33. Practice Estimating Causal E↵ects Using Linear Regression Hypothesis 2: Considering trend e↵ect Estimate Std. Error t value Pr(>|t|) (Intercept) 1.244e-02 1.416e-03 8.782 1.15e-09 eVote -1.890e-02 1.270e-03 -14.881 4.12e-15 Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 23 / 38
  34. 34. Practice Estimating Causal E↵ects Using Linear Regression Compared the models H Two Points All with Trend H1 Coef. 0.0535 0.0247 0.0240 S.E. (0.0055) (0.0062) (0.0054) H2 Coef. -0.0165 -0.0188 -0.0189 S.E. (0.0011) (0.0013) (0.0013) ) The estimates of H1 are less stable than that of H2 Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 24 / 38
  35. 35. Practice Estimating Causal E↵ects Using Linear Regression Visualization of Di↵-in-Di↵(All the Points)0.250.300.350.400.450.50 Year VoterTurnout 2000 2004 2008 2012 2016 Adoptaion e-Vote (Higashiyama) Adoptaion e-Vote (Kamikyo) Abolishing e-Vote (Both) Kamikyo Higashiyama Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 25 / 38
  36. 36. Practice Estimating Causal E↵ects Using Linear Regression Visualization of Di↵-in-Di↵(All the Points)0.0000.0050.0100.0150.0200.0250.030 Year SpoiltVotes 2000 2004 2008 2012 2016 Adoptaion e-Vote (Higashiyama) Adoptaion e-Vote (Kamikyo) Abolishing e-Vote (Both) Kamikyo Higashiyama Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 25 / 38
  37. 37. Standard Errors in Di↵-in-Di↵ Estimation How to Calculate S.Es Clustered standard errors can help us. These can be easily calculated using multiwayvcov and lmtest packages. (We can conduct Di↵-in-Di↵ with adjusted standard errors using R package, wfe, but it does not work on my PC.) Let’s try to calculate clustered standard errors of Hypothesis 2(spoilt votes). Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 26 / 38
  38. 38. Standard Errors in Di↵-in-Di↵ Estimation Calculate Clustered Standard Errors: Code # Load required packages library(multiwayvcov) library(lmtest) # Calculate the clustered var-cov matrix H2Model3_VCOV <- cluster.vcov(H2Model3, ~WardID) # PROFIT! coeftest(H2Model3, H2Model3_VCOV) Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 27 / 38
  39. 39. Standard Errors in Di↵-in-Di↵ Estimation Calculate Clustered Standard Errors: Result without clustering t test of coefficients: Estimate Std. Error t value Pr(>|t|) eVote -1.890e-02 1.270-03 -14.881 < 2.2e-16 with clustering t test of coefficients: Estimate Std. Error t value Pr(>|t|) eVote -1.890e-02 2.0491e-03 -9.2235 < 2.2e-16 Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 28 / 38
  40. 40. Synthetic Control Method Introduction Objective: to evaluate the impact of a treatment implemented at the aggregate level (e.g. country, region) on one or few units using a small number of controls to build the counterfactual Synthetic control methods use panel data to build the weighted average of non-treated units that best reproduces characteristics of the treated unit over time impact of the treatment is measured by a simple di↵erence after treatment between the treated and a combination of comparison units(synthetic control) Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 29 / 38
  41. 41. Synthetic Control Method Setup Units: j = 1, 2, . . . , J + 1 where j = 1 is the treated and j = 2, . . . , J + 1 are controls (potential comparisons) Time frame:split t = 1, . . . , T1 into two periods,pretreatment t = 1, . . . , T0 and post-treatment t = T0 + 1, . . . , T1 Potential and observed outcomes for the treated unit are (Y 0 1t, Y 1 1t) where Y1t = ( Y 0 1t t = 1, . . . , T0 Y 1 1t t = T0 + 1, . . . , T1 Our objective is to estimate ↵1t = Y 1 1t Y 0 1t Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 30 / 38
  42. 42. Synthetic Control Method Setup,continued Let X1 be a k ⇥ 1 vector of pre-intervention characteristics of the treated units Let X0 is a k ⇥ J vector of the same variables for the comparison units Choose weights that minimize kX m=1 vm(X1m X0mW)2 where X1m is the value of the m-th variable for the treated,vm is a weight that reflects the relative importance that we assign to the m-th variable Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 31 / 38
  43. 43. Synthetic Control Method Setup,continued Choose W⇤ = (w⇤ 2, . . . , wJ + 1⇤ ) 2 [0, 1]J ,adding to 1 to minimize distance in pretreatment characteristics between treated and weighted average of controls Treatment e↵ect estimated by the simple di↵erence ˆ↵1t = Y 1 1t PJ+1 j=2 w⇤ j Yjt for t = T0 + 1, . . . , T1 Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 32 / 38
  44. 44. Synthetic Control Method Application,German Unification This paper aims to examine the e↵ect of the 1990 German reunification on per capita GDP in West Germany the set of comparisons is a sample of OECD countries Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 33 / 38
  45. 45. Synthetic Control Method Predictors of Economic Growth Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 34 / 38
  46. 46. Synthetic Control Method West Germany and synthetic West Germany Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 35 / 38
  47. 47. Synthetic Control Method Per Capita GDP gap Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 36 / 38
  48. 48. Synthetic Control Method Placebo Studies In-time Placebo:apply this method to dates when the intervention didn’t occur In-space Placebo: resign the intervention to a comparison unit Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 37 / 38
  49. 49. Synthetic Control Method Robustness Checks Test the sensitivity of the main results to changes in the country weights. Incorporate the leave-one-out estimates Jia Li, Jaehyun Song (Kobe Univ.) Di↵-in-Di↵ 2016-07-27 38 / 38

×