Within Models

1. Peter M. Lance, PhD MEASURE Evaluation University of North Carolina at Chapel Hill December 15, 2016 Within Models

2. Global, five-year, $180M cooperative agreement Strategic objective: To strengthen health information systems – the capacity to gather, interpret, and use data – so countries can make better decisions and sustain good health outcomes over time. Project overview

3. Improved country capacity to manage health information systems, resources, and staff Strengthened collection, analysis, and use of routine health data Methods, tools, and approaches improved and applied to address health information challenges and gaps Increased capacity for rigorous evaluation Phase IV Results Framework

4. Global footprint (more than 25 countries)

5. • The program impact evaluation challenge • Randomization • Selection on observables • Within estimators • Instrumental variables

6. • The program impact evaluation challenge • Randomization • Selection on observables • Within estimators • Instrumental variables

7. 𝑌0 = 𝛽0 + 𝛽2 ∙ 𝑥 + 𝜀 𝑌1 = 𝛽0 + 𝛽1 + 𝛽2 ∙ 𝑥 + 𝜀

8. 𝑌1 − 𝑌0 = 𝛽0 + 𝛽1 + 𝜖 − 𝛽0 + 𝜖 = 𝛽0 + 𝛽1 + 𝜖 − 𝛽0 − 𝜖 = 𝛽1

9. 𝑌1 − 𝑌0 = 𝛽0 + 𝛽1 + 𝛽2 ∙ 𝑥 + 𝜀 − 𝛽0 + 𝛽2 ∙ 𝑥 + 𝜀 = 𝛽0 + 𝛽1 + 𝜖 − 𝛽0 − 𝜖 = 𝛽1

10. 𝑌1 − 𝑌0 = 𝛽1

11. 𝑌 = 𝑃 ∙ 𝑌1 + 1 − 𝑃 ∙ 𝑌0 = 𝑃 ∗ 𝛽0 + 𝛽1 + 𝜖 + 1 − 𝑃 ∗ 𝛽0 + 𝜖 = 𝑃 ∗ 𝛽0 + 𝑃 ∗ 𝛽1 + 𝑃 ∗ 𝜖 +𝛽0 + 𝜖 − 𝑃 ∗ 𝛽0 − 𝑃 ∗ 𝜖 = 𝛽0 + 𝑃 ∗ 𝛽1 + 𝜖

12. 𝑌 = 𝑃 ∙ 𝑌1 + 1 − 𝑃 ∙ 𝑌0 = 𝑃 ∙ 𝛽0 + 𝛽1 + 𝛽2 ∙ 𝑥 + 𝜀 + 1 − 𝑃 ∙ 𝛽0 + 𝛽2 ∙ 𝑥 + 𝜀 = 𝑃 ∗ 𝛽0 + 𝑃 ∗ 𝛽1 + 𝑃 ∗ 𝜖 +𝛽0 + 𝜖 − 𝑃 ∗ 𝛽0 − 𝑃 ∗ 𝜖

13. 𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝜀

14. Cost of Participation 𝐶 = 𝜌0 + 𝜌1 ∙ 𝑥

15. Cost of Participation 𝐶 = 𝜌0 + 𝜌1 ∙ 𝑥

16. Benefit-Cost>0 𝑌1 − 𝑌0 − C > 0 𝛽1 − 𝐶 > 0 𝛽1 − 𝛾0 + 𝛾1 ∗ 𝑥 > 0

19. Benefit-Cost>0 𝑌1 − 𝑌0 − C > 0 𝛽1 − 𝐶 > 0 𝛽1 − 𝜌0 + 𝜌1 ∙ 𝑥 > 0

20. 𝑥 𝑃

21. Benefit-Cost>0 𝛽1 − 𝜌0 + 𝜌1 ∙ 𝑥 > 0 𝜀

24. P and 𝜺 are independent

25. True model: 𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝜀 We actually attempt to estimate: 𝑌 = 𝜏0 + 𝜏1 ∙ 𝑃 + 𝜖 Error term now contains: 𝛽2 ∙ 𝑥

28. 𝐸( 𝜏1) = 𝐸 𝑖=1 𝑛 𝑃𝑖 − 𝑃 ∙ 𝑌𝑖 𝑖=1 𝑛 𝑃𝑖 − 𝑃 2

29. 𝐸 𝜏1 = 𝐸 𝑖=1 𝑛 𝑃𝑖 − 𝑃 ∙ 𝛽0 + 𝛽1 ∙ 𝑃𝑖 + 𝛽2 ∙ 𝑥𝑖 + 𝜀𝑖 𝑖=1 𝑛 𝑃𝑖 − 𝑃 2

30. 𝐸 𝜏1 = 𝛽1 +𝐸 𝑖=1 𝑛 𝛽2 ∙ 𝑥𝑖 ∙ 𝑃𝑖 − 𝑃 𝑖=1 𝑛 𝑃𝑖 − 𝑃 2

31. 𝐸 𝜏1 = 𝛽1 +𝐸 𝑖=1 𝑛 𝛽2 ∙ 𝑥𝑖 ∙ 𝑃𝑖 − 𝑃 𝑖=1 𝑛 𝑃𝑖 − 𝑃 2

32. 𝐸 𝜏1 = 𝛽1 +𝛽2 ∙ 𝐸 𝑖=1 𝑛 𝑥𝑖 ∙ 𝑃𝑖 − 𝑃 𝑖=1 𝑛 𝑃𝑖 − 𝑃 2

33. 𝐸 𝜏1 = 𝛽1 +𝛽2 ∙ 𝐸 𝑖=1 𝑛 𝑥𝑖 ∙ 𝑃𝑖 − 𝑃 𝑖=1 𝑛 𝑃𝑖 − 𝑃 2

34. 𝐸 𝜏1 = 𝛽1 +𝛽2 ∙ 𝐸 𝑖=1 𝑛 𝑥𝑖 ∙ 𝑃𝑖 − 𝑃 𝑖=1 𝑛 𝑃𝑖 − 𝑃 2 𝑥𝑖= 𝛾0 + 𝛾1 ∙ 𝑃𝑖 + 𝜗𝑖

37. 𝐸 𝜏1 = 𝛽1 +𝛽2 ∙ 𝛾1 𝑖=1 𝑛 𝑥 𝑖=1 𝑛 𝑥 𝑥𝑖= 𝛾0 + 𝛾1 ∙ 𝑃𝑖 + 𝜗𝑖

39. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝐸 𝑖=1 𝑛 𝑥𝑖 ∙ 𝑃𝑖 − 𝑃 𝑖=1 𝑛 𝑃𝑖 − 𝑃 2

40. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝛾1

41. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝛾1 The actual causal effect of P on y

42. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝛾1 The actual causal effect of P on y The actual causal effect of the omitted variable X on Y

43. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝛾1 The actual causal effect of P on y Th”Effect” of P on x: 𝑥𝑖= 𝛾0 + 𝛾1 ∙ 𝑃𝑖 + 𝜗𝑖 The actual causal effect of the omitted variable X on Y

48. True model: 𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝜀 We actually attempt to estimate: 𝑌 = 𝜏0 + 𝜏1 ∙ 𝑃 + 𝜖 𝑬 𝝉 𝟏 ≠ 𝜷

49. 𝐸 𝜏1 = 𝛽1 + 𝛽2 ∙ 𝛾1 The actual causal effect of P on y Th”Effect” of P on x: 𝑥𝑖= 𝛾0 + 𝛾1 ∙ 𝑃𝑖 + 𝜗𝑖 The actual causal effect of the omitted variable X on Y

50. X Y P

51. X Y P

53. X, µ Y P

54. True model: 𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝛽3 ∙ 𝜇 + 𝜀 We actually attempt to estimate: 𝑌 = 𝜏0 + 𝜏1 ∙ 𝑃 + 𝜏2 ∙ 𝑥 + 𝜖 Error term now contains: 𝜇

55. Cost of articipation 𝐶 = 𝜌0 + 𝜌1 ∙ 𝑥 + 𝜌2 ∙ 𝜇

56. Benefit-Cost>0 𝑌1 − 𝑌0 − C > 0 𝛽1 − 𝐶 > 0 𝛽1 − 𝜌0 + 𝜌1 ∙ 𝑥 + 𝜌2 ∙ 𝜇 > 0

57. 𝑥 𝑃 𝜇 𝑃

58. 𝑥 𝑃 𝜇 𝑃

59. True model: 𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝛽3 ∙ 𝜇 + 𝜀 We actually attempt to estimate: 𝑌 = 𝜏0 + 𝜏1 ∙ 𝑃 + 𝜏2 ∙ 𝑥 + 𝜖 Error term now contains: 𝜇

60. (John) WHAZZUP!!!!

61. (Ben) This webinar is… simply terrible (John)

62. (John) (Ben)

63. (John) (Ben) 𝑃 = 1 𝑌𝐵𝑒𝑛 = 𝑌𝐵𝑒𝑛 1 𝑊𝑒𝑎𝑙𝑡ℎ𝑦 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑 𝑃 = 0 𝑌𝐽𝑜ℎ𝑛 = 𝑌𝐽𝑜ℎ𝑛 0 𝑃𝑜𝑜𝑟 𝑈𝑛 − 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑

64. X, µ Y P

65. (John) (Ben) 𝑃 = 1 𝑌𝐵𝑒𝑛 = 𝑌𝐵𝑒𝑛 1 𝑊𝑒𝑎𝑙𝑡ℎ𝑦 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑 𝑃 = 0 𝑌𝐽𝑜ℎ𝑛 = 𝑌𝐽𝑜ℎ𝑛 0 𝑃𝑜𝑜𝑟 𝑈𝑛 − 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑 𝒀 𝑩𝒆𝒏 − 𝒀 𝑱𝒐𝒉𝒏

66. (John) (Ben) 𝑃 = 1 𝑌𝐵𝑒𝑛 = 𝑌𝐵𝑒𝑛 1 𝑊𝑒𝑎𝑙𝑡ℎ𝑦 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑 𝑃 = 0 𝑌𝐽𝑜ℎ𝑛 = 𝑌𝐽𝑜ℎ𝑛 0 𝑃𝑜𝑜𝑟 𝑈𝑛 − 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑 𝒀 𝑩𝒆𝒏 − 𝒀 𝑱𝒐𝒉𝒏

67. (John) (Ben) 𝑃 = 1 𝑌𝐵𝑒𝑛 = 𝑌𝐵𝑒𝑛 1 𝑊𝑒𝑎𝑙𝑡ℎ𝑦 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑 𝑃 = 0 𝑌𝐽𝑜ℎ𝑛 = 𝑌𝐽𝑜ℎ𝑛 0 𝑃𝑜𝑜𝑟 𝑈𝑛 − 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑 𝒀 𝑩𝒆𝒏 − 𝒀 𝑱𝒐𝒉𝒏 ?

68. True model: 𝑌 = 𝛽0 + 𝛽1 ∙ 𝑃 + 𝛽2 ∙ 𝑥 + 𝛽3 ∙ 𝜇 + 𝜀 We actually attempt to estimate: 𝑌 = 𝜏0 + 𝜏1 ∙ 𝑃 + 𝜖 Error term now contains: 𝛽2 ∙ 𝑥 + 𝛽3 ∙ 𝜇

69. (John) (Ben) 𝑃 = 1 𝑌𝐵𝑒𝑛 = 𝑌𝐵𝑒𝑛 1 𝑊𝑒𝑎𝑙𝑡ℎ𝑦 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑 𝑃 = 0 𝑌𝐽𝑜ℎ𝑛 = 𝑌𝐽𝑜ℎ𝑛 0 𝑃𝑜𝑜𝑟 𝑈𝑛 − 𝑀𝑜𝑡𝑖𝑣𝑎𝑡𝑒𝑑 𝒀 𝑩𝒆𝒏 − 𝒀 𝑱𝒐𝒉𝒏 ?

71. P t

72. P t 1 0

73. P t 1 0

74. P t 1 0

75. P t 1 0

76. X, µ Y P

77. 𝑃𝐵𝑒𝑛,1 𝑌𝐵𝑒𝑛,1𝜇 𝐵𝑒𝑛 𝑡 = 1 𝑃𝐵𝑒𝑛,0 𝑌𝐵𝑒𝑛,0𝜇 𝐵𝑒𝑛 𝑡 = 0

78. 𝑃𝐵𝑒𝑛,1 𝑌𝐵𝑒𝑛,1𝜇 𝐵𝑒𝑛 𝑡 = 1 𝑃𝐵𝑒𝑛,0 𝑌𝐵𝑒𝑛,0𝜇 𝐵𝑒𝑛 𝑡 = 0

79. 𝑃𝐵𝑒𝑛,1 𝑌𝐵𝑒𝑛,1𝜇 𝐵𝑒𝑛 𝑃𝐵𝑒𝑛,0 𝑌𝐵𝑒𝑛,0𝜇 𝐵𝑒𝑛

80. 𝑃𝐵𝑒𝑛,1 𝑌𝐵𝑒𝑛,1𝜇 𝐵𝑒𝑛 𝑃𝐵𝑒𝑛,0 𝑌𝐵𝑒𝑛,0𝜇 𝐵𝑒𝑛 𝑃𝐵𝑒𝑛,1 − 𝑃𝐵𝑒𝑛,0

81. 𝑃𝐵𝑒𝑛,1 𝑌𝐵𝑒𝑛,1𝜇 𝐵𝑒𝑛 𝑃𝐵𝑒𝑛,0 𝑌𝐵𝑒𝑛,0𝜇 𝐵𝑒𝑛 ∆𝑃𝐵𝑒𝑛

82. 𝑃𝐵𝑒𝑛,1 𝑌𝐵𝑒𝑛,1𝜇 𝐵𝑒𝑛 𝑃𝐵𝑒𝑛,0 𝑌𝐵𝑒𝑛,0𝜇 𝐵𝑒𝑛 ∆𝑃𝐵𝑒𝑛 ∆𝑌𝐵𝑒𝑛

83. 𝑃𝐵𝑒𝑛,1 𝑌𝐵𝑒𝑛,1𝜇 𝐵𝑒𝑛 𝑃𝐵𝑒𝑛,0 𝑌𝐵𝑒𝑛,0𝜇 𝐵𝑒𝑛 ∆𝑃𝐵𝑒𝑛 ∆𝑌𝐵𝑒𝑛𝜇 𝐵𝑒𝑛 − 𝜇 𝐵𝑒𝑛

84. 𝑃𝐵𝑒𝑛,1 𝑌𝐵𝑒𝑛,1𝜇 𝐵𝑒𝑛 𝑃𝐵𝑒𝑛,0 𝑌𝐵𝑒𝑛,0𝜇 𝐵𝑒𝑛 ∆𝑃𝐵𝑒𝑛 ∆𝑌𝐵𝑒𝑛0

87. 𝑃𝐵𝑒𝑛,𝑡, 𝑌𝐵𝑒𝑛,𝑡𝜇 𝐵𝑒𝑛 ∆𝑃𝐵𝑒𝑛, ∆𝑌𝐵𝑒𝑛 𝜇 𝐵𝑒𝑛

88. Later folks!!! It was awesome!! I…deeply regret my role in this webinar

89. 𝑌𝑖𝑡 0 = 𝛽0 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 1 = 𝛽0 + 𝛽1 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡

90. 𝑌𝑖𝑡 1 − 𝑌𝑖𝑡 0 = 𝛽0 + 𝛽1 + 𝜖 − 𝛽0 − 𝜖 = 𝛽1

91. 𝑌𝑖𝑡 1 − 𝑌𝑖𝑡 0 = 𝛽0 + 𝛽1 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 − 𝛽0 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 = 𝛽0 + 𝛽1 + 𝜖 − 𝛽0 − 𝜖 = 𝛽1

92. 𝑌𝑖𝑡 1 − 𝑌𝑖𝑡 0 = 𝛽1 = 𝛽0 + 𝛽1 + 𝜖 − 𝛽0 − 𝜖 = 𝛽1

93. 𝑌𝑖𝑡 = 𝑃𝑖𝑡 ∙ 𝑌𝑖𝑡 1 + 1 − 𝑃𝑖𝑡 ∙ 𝑌𝑖𝑡 0 = 𝑃𝑖𝑡 ∙ 𝛽0 + 𝛽1 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 + 1 − 𝑃𝑖𝑡 ∙ 𝛽0 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 = 𝛽0 + 𝑃 ∗ 𝛽1 + 𝜖

94. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡

95. Cost of Participation 𝐶𝑖𝑡 = 𝜌0 + 𝜌1 ∙ 𝑥𝑖𝑡 + 𝜌2 ∙ 𝜇𝑖

96. Benefitit-Costit>0 𝑌𝑖𝑡 1 − 𝑌𝑖𝑡 0 − 𝐶𝑖𝑡 > 0 𝛽1 − 𝐶𝑖𝑡 > 0 𝛽1 − 𝜌0 + 𝜌1 ∙ 𝑥𝑖𝑡 + 𝜌2 ∙ 𝜇𝑖 > 0

99. 𝑥𝑖𝑡 𝑃𝑖𝑡

100. 𝜇𝑖 𝑃𝑖𝑡

101. X, µ Y P

102. (The Truth) 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 (What we can actually estimate) 𝑌𝑖𝑡 = 𝛾0 + 𝛾1 ∙ 𝑃𝑖𝑡 + 𝛾2 ∙ 𝑥𝑖𝑡 + 𝜖𝑖𝑡 (The $60,000 Question) 𝐸 𝛾1 = 𝛽1 ?

106. (The Truth) 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 (What we can actually estimate) 𝑌𝑖𝑡 = 𝛾0 + 𝛾1 ∙ 𝑃𝑖𝑡 + 𝛾2 ∙ 𝑥𝑖𝑡 + 𝜖𝑖𝑡 (The $60,000 Question) 𝐸 𝛾1 ≠ 𝛽1

107. Uh Oh

108. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 Time (t) t=0 t=1

109. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 Time (t) t=0 t=1

110. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 Time (t) t=0 t=1

111. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0

112. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 𝑌𝑖1 − 𝑌𝑖0

113. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖

114. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽0 − 𝛽0

115. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 0

116. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖

117. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖

118. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + 𝛽3 ∙ 𝜇𝑖 − 𝜇𝑖

119. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + 𝛽3 ∙ 0

120. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + ∆𝜀𝑖

121. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + ∆𝜀𝑖

122. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + ∆𝜀𝑖 𝜇𝑖

125. 𝑌𝑖𝑡 0 = 𝛽0 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 1 = 𝛽0 + 𝛽1 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡

126. 𝑌𝑖𝑡 0 = 𝛽0 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 1 = 𝛽0 + 𝛽1 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝛽00 ∙ 𝑡

128. ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + ∆𝜀𝑖 Time (t)

129. ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + ∆𝜀𝑖 Time (t)

130. ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + ∆𝜀𝑖 Time (t) t=1 t=2 ∙∙∙∙∙∙∙ t=T-1 t=T

131. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡

132. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 = 𝑌𝑖𝑡 − 𝑌𝑖 𝑌𝑖 = 𝑡=1 𝑇 𝑌𝑖𝑡 𝑇 𝑌𝑖𝑡=

133. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 = 𝑌𝑖𝑡 − 𝑌𝑖 𝑌𝑖 = 𝑡=1 𝑇 𝑌𝑖 𝑇 𝑌𝑖𝑡= 0

134. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑃𝑖𝑡 = 𝑃𝑖𝑡 − 𝑃𝑖 𝑃𝑖 = 𝑡=1 𝑇 𝑃𝑖 𝑇 𝑌𝑖𝑡= 𝛽1 ∙ 𝑃𝑖𝑡

135. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑥𝑖𝑡 = 𝑥𝑖𝑡 − 𝑥𝑖 𝑥𝑖 = 𝑡=1 𝑇 𝑥𝑖 𝑇 𝑌𝑖𝑡= 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡

136. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝜇𝑖𝑡 = 𝜇𝑖 − 𝜇𝑖 = 𝜇𝑖 − 𝜇𝑖 = 0 𝜇𝑖 = 𝑡=1 𝑇 𝜇𝑖 𝑇 = 𝜇𝑖 𝑌𝑖𝑡= 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 0

137. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝜀𝑖𝑡 = 𝜀𝑖𝑡 − 𝜀𝑖 𝜀𝑖 = 𝑡=1 𝑇 𝜀𝑖 𝑇 𝑌𝑖𝑡= 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝜀𝑖𝑡

138. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝑖=1 𝑁 𝑑𝑖 ∙ 𝜇𝑖 + 𝜀𝑖𝑡

139. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝑗=1 𝑁 𝑑𝑗 ∙ 𝜇 𝑗 + 𝜀𝑖𝑡

140. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝑗=1 𝑁 𝛽3 ∙ 𝑑𝑗 ∙ 𝜇 𝑗 + 𝜀𝑖𝑡

141. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝑗=1 𝑁 𝑑𝑗 ∙ 𝛽3∙ 𝜇 𝑗 + 𝜀𝑖𝑡

142. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝑗=1 𝑁 𝑑𝑗 ∙ 𝜑𝑗 + 𝜀𝑖𝑡 where 𝜑𝑗 = 𝛽3 ∙ 𝜇 𝑗

143. 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖𝑡 𝑌𝑖𝑡 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖𝑡 + 𝛽2 ∙ 𝑥𝑖𝑡 + 𝑗=1 𝑁−1 𝑑𝑗 ∙ 𝜑𝑗 + 𝜀𝑖𝑡 where 𝜑𝑗 = 𝛽3 ∙ 𝜇 𝑗

144. Big Caveats/Limitations/Drawbacks 1.Loss of information 2.Makes measurement error bias worse 3.Very limited options for limited dependent variables 4.Rooted in a weird kind of paradox

146. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + ∆𝜀𝑖 (1) (2)

148. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖1 = 𝛾0 + 𝛾1 ∙ 𝑃𝑖1 + 𝛾2 ∙ 𝑥𝑖1 + 𝛾3 ∙ 𝜇𝑖 + 𝜖𝑖1 where 𝑃𝑖1 = 𝑃𝑖1 + 𝜏𝑖1 𝐸 𝛾1 ≠ 𝛽1

151. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖1 = 𝛾0 + 𝛾1 ∙ 𝑃𝑖1 + 𝛾2 ∙ 𝑥𝑖1 + 𝛾3 ∙ 𝜇𝑖 + 𝜖𝑖1 where 𝑃𝑖1 = 𝑃𝑖1 + 𝜏𝑖1 𝐸 𝛾1 < 𝛽1

152. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖1 = 𝛾0 + 𝛾1 ∙ 𝑃𝑖1 + 𝛾2 ∙ 𝑥𝑖1 + 𝛾3 ∙ 𝜇𝑖 + 𝜖𝑖1 where 𝑃𝑖1 = 𝑃𝑖1 + 𝜏𝑖1 𝐸 𝛾1 < 𝛽1

153. 𝑌𝑖1 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖1 + 𝛽2 ∙ 𝑥𝑖1 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖1 𝑌𝑖0 = 𝛽0 + 𝛽1 ∙ 𝑃𝑖0 + 𝛽2 ∙ 𝑥𝑖0 + 𝛽3 ∙ 𝜇𝑖 + 𝜀𝑖0 ∆𝑌𝑖 = 𝛽1 ∙ ∆𝑃𝑖 + 𝛽2 ∙ ∆𝑥𝑖 + ∆𝜀𝑖 𝑃𝑖 ∆𝑃𝑖

154. t=2000 t=2002 True age 35 37 Measured age 33 39 Error -2 +2 Error 𝐓𝐫𝐮𝐞 𝐚𝐠𝐞 .0571 .054 Age2002-Age2000 True age difference 2 Measured age difference 6 Error 4 Error 𝐓𝐫𝐮𝐞 difference 1.5

157. Difference-in-Differences

158. time (t)

159. time (t)t=0

160. Village 1 Village 2 t=0 time (t)t=0

161. Village 1 Village 2 t=0 time (t)t=0

162. Village 1 Village 2 t=0 time (t)t=0 P=0 P=1

163. Village 1 Village 2 Village 1 Village 2 t=0 t=1 time (t)t=0 t=1

164. Village 1 Village 2 Village 1 Village 2 t=0 t=1 time (t)t=0 t=1

165. Village 1 Village 2 Village 1 Village 2 t=0 t=1 time (t)t=0 t=1 P=0 P=1

166. Village 1 Village 2 Village 1 Village 2 t=0 t=1 time (t)t=0 t=1 P=0 P=1

167. Y t

168. Y tt=0 t=1

169. Y tt=0 t=1 E(Y|P=0,t=0)

170. Y tt=0 t=1 E(Y|P=0,t=0) E(Y|P=0,t=1)

171. Y tt=0 t=1 E(Y|P=0,t=0) E(Y|P=0,t=1) A A=E(Y|P=0,t=1)-E(Y|P=0,t=0)

172. Y tt=0 t=1 E(Y|P=0,t=0) E(Y|P=0,t=1) E(Y|P=1,t=0) A

173. Y tt=0 t=1 E(Y|P=0,t=0) E(Y|P=0,t=1) E(Y|P=1,t=0) E(Y|P=1,t=1) A

176. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜖𝑖𝑡

177. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜖𝑖𝑡 Controls for fixed (ie underlying, not time-varying) differences between program participants and non-participants

179. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜖𝑖𝑡 Controls for fixed (ie underlying, not time-varying) differences between program participants and non-participants Controls for underlying time trend common to program participants and non-participants

180. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜖𝑖𝑡 Controls for fixed (ie underlying, not time-varying) differences between program participants and non-participants Controls for underlying time trend common to program participants and non-participants

181. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜖𝑖𝑡 Controls for fixed (ie underlying, not time-varying) differences between program participants and non-participants Controls for underlying time trend common to program participants and non-participants Program impact

182. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜔4 ∙ 𝑋𝑖𝑡 + 𝜖𝑖𝑡 Controls for other time-varying characteristics

183. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜔4 ∙ 𝑋𝑖𝑡 + 𝜖𝑖𝑡 𝑐𝑜𝑟𝑟 𝜖𝑖𝑡, 𝜖𝑖𝑡+𝑗 ≠ 0

184. Bertrand, Duflo and Mullainathan Basic Experiment: Take a dataset (current population survey) with labor market outcomes (ln(earnings)) for many women-years (900,000) and “make up” a fake program. Then try to evaluate the impact of these fake programs with a DID regression. The Result: The null hypothesis that the policy had no effect at the 5 percent level rejected a stunning 50-70 percent of the time, depending on the econometric approach.

185. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑃𝑖 + 𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 + 𝜔4 ∙ 𝑋𝑖𝑡 + 𝜖𝑖𝑡 The “group mean” fixed effect

186. 𝑌𝑖𝑡 = 𝜔0 + 𝜔1 ∙ 𝑖=1 𝑁 𝑑𝑖 ∙ 𝜇𝑖 +𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 +𝜔4 ∙ 𝑋𝑖𝑡 + 𝜖𝑖𝑡 Dummy variable individual fixed effect

187. 𝑌𝑖𝑡 = 𝜔0 + 𝑖=1 𝑁 𝑑𝑖 ∙ 𝜔1 ∙ 𝜇𝑖 +𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 +𝜔4 ∙ 𝑋𝑖𝑡 + 𝜖𝑖𝑡

188. 𝑌𝑖𝑡 = 𝜔0 + 𝑖=1 𝑁 𝑑𝑖 ∙ 𝜔1𝑖 +𝜔2 ∙ 𝑡 + 𝜔3 ∙ 𝑃𝑖 ∙ 𝑡 +𝜔4 ∙ 𝑋𝑖𝑡 + 𝜖𝑖𝑡 where 𝜔1𝑖 = 𝜔1 ∙ 𝜇𝑖

189. Conclusion

190. Links: The manual: http://www.measureevaluation.org/resources/publications/ms- 14-87-en The webinar introducing the manual: http://www.measureevaluation.org/resources/webinars/metho ds-for-program-impact-evaluation My email: pmlance@email.unc.edu

191. MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) under terms of Cooperative Agreement AID-OAA-L-14-00004 and implemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International, John Snow, Inc., Management Sciences for Health, Palladium Group, and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government. www.measureevaluation.org

Within Models

You are reading a preview.

Check these out next

Within Models

More Related Content

Viewers also liked (20)

Similar to Within Models (20)

More from MEASURE Evaluation (20)

Recently uploaded (20)

Within Models

Within Models

You are reading a preview.

Within Models

Recommended

More Related Content

Viewers also liked (20)

Similar to Within Models (20)

More from MEASURE Evaluation (20)

Recently uploaded (20)

Within Models