Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Measuring Impact Qualitatively by MEASURE Evaluation 3098 views
- An Introduction to the Manual:How ... by MEASURE Evaluation 2712 views
- Impact Evaluation Overview by Philip Jakob 309 views
- How we can use impact evaluation to... by IFAD Internationa... 10214 views
- Project Management Cycle: Tackling ... by MEASURE Evaluation 2318 views
- Impact Evaluation, Policy Making an... by LIDC 1701 views

2,445 views

Published on

Published in:
Education

No Downloads

Total views

2,445

On SlideShare

0

From Embeds

0

Number of Embeds

16

Shares

0

Downloads

591

Comments

0

Likes

9

No embeds

No notes for slide

- 1. Peter M. Lance, PhD MEASURE Evaluation University of North Carolina at Chapel Hill MARCH 31, 2016 Fundamentals of Program Impact Evaluation
- 2. Global, five-year, $180M cooperative agreement Strategic objective: To strengthen health information systems – the capacity to gather, interpret, and use data – so countries can make better decisions and sustain good health outcomes over time. Project overview
- 3. Improved country capacity to manage health information systems, resources, and staff Strengthened collection, analysis, and use of routine health data Methods, tools, and approaches improved and applied to address health information challenges and gaps Increased capacity for rigorous evaluation Phase IV Results Framework
- 4. Global footprint (more than 25 countries)
- 5. How Do We Know IfAProgram MadeADifference? ABrief Helicopter Tour of Methods for Estimating Program Impact
- 6. • The Program Impact Evaluation Challenge • Randomization • Selection on observables • Within estimators • Instrumental variables
- 7. • The Program Impact Evaluation Challenge • Randomization • Selection on observables • Within estimators • Instrumental variables
- 8. • The Program Impact Evaluation Challenge • Randomization • Selection on observables • Within estimators • Instrumental variables
- 9. • The Program Impact Evaluation Challenge • Randomization • Selection on observables • Within estimators • Instrumental variables
- 10. • The Program Impact Evaluation Challenge • Randomization • Selection on observables • Within estimators • Instrumental variables
- 11. • The Program Impact Evaluation Challenge • Randomization • Selection on observables • Within estimators • Instrumental variables
- 12. • The Program Impact Evaluation Challenge • Randomization • Selection on observables • Within estimators • Instrumental variables
- 13. Newton’s “Laws” of Motion 𝐹𝑜𝑟𝑐𝑒 = 𝑀𝑎𝑠𝑠 ∙ 𝐴𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛
- 14. Did the program make a difference?
- 15. Did the program cause a change in an outcome of interest Y ? (Causality)
- 16. Our outcome of Interest What happens if an individual does not participate in a program What happens if that individual does participate in a program Potential Outcomes 𝑌 : 𝑌0 : 𝑌1 :
- 17. Our outcome of interest What happens if an individual does not participate in a program What happens if that individual does participate in a program Potential Outcomes 𝑌𝑖 : 𝑌𝑖 0 : 𝑌𝑖 1 :
- 18. Our outcome of interest What happens if an individual does not participate in a program What happens if that individual does participate in a program Potential Outcomes 𝑌𝑖 : 𝑌𝑖 0 : 𝑌𝑖 1 :
- 19. Our outcome of interest What happens if an individual does not participate in a program What happens if that individual does participate in a program Potential Outcomes 𝑌𝑖 : 𝑌𝑖 0 : 𝑌𝑖 1 :
- 20. What happens if the individual participates {Causal} Program Impact 𝑌𝑖 1 − 𝑌𝑖 0 = Program Impact What happens if the individual does not participate
- 21. What happens if the individual participates {Causal} Program Impact 𝑌𝑖 1 − 𝑌𝑖 0 = Program Impact What happens if the individual does not participate
- 22. What happens if the individual participates {Causal} Program Impact 𝑌𝑖 1 − 𝑌𝑖 0 = Program Impact What happens if the individual does not participate
- 23. What happens if the individual participates {Causal} Program Impact 𝑌𝑖 1 − 𝑌𝑖 0 = Program Impact What happens if the individual does not participate
- 24. What happens if the individual participates {Causal} Program Impact 𝑌𝑖 1 − 𝑌𝑖 0 = Program Impact What happens if the individual does not participate
- 25. 𝑃𝑖 = 1ifindividual 𝑖 participates 0if individual 𝑖 does not participate Program Participation
- 26. 𝑌𝑖 = 𝑃𝑖 ∙ 𝑌𝑖 1 + 1 − 𝑃𝑖 ∙ 𝑌𝑖 0 Observed Outcome
- 27. 𝑌𝑖 = 𝑃𝑖 ∙ 𝑌𝑖 1 + 1 − 𝑃𝑖 ∙ 𝑌𝑖 0 Observed Outcome 𝑃𝑖 = 1
- 28. 𝑌𝑖 = 1 ∙ 𝑌𝑖 1 + 1 − 1 ∙ 𝑌𝑖 0 Observed Outcome 𝑃𝑖 = 1
- 29. 𝑌𝑖 = 𝑌𝑖 1 + 0 ∙ 𝑌𝑖 0 Observed Outcome 𝑃𝑖 = 1
- 30. 𝑌𝑖 = 𝑌𝑖 1 Observed Outcome 𝑃𝑖 = 1
- 31. 𝑌𝑖 = 𝑃𝑖 ∙ 𝑌𝑖 1 + 1 − 𝑃𝑖 ∙ 𝑌𝑖 0 Observed Outcome
- 32. 𝑌𝑖 = 𝑃𝑖 ∙ 𝑌𝑖 1 + 1 − 𝑃𝑖 ∙ 𝑌𝑖 0 Observed Outcome 𝑃𝑖 = 0
- 33. 𝑌𝑖 = 0 ∙ 𝑌𝑖 1 + 1 − 0 ∙ 𝑌𝑖 0 Observed Outcome 𝑃𝑖 = 0
- 34. 𝑌𝑖 = 𝑌𝑖 0 Observed Outcome 𝑃𝑖 = 0
- 35. 𝑌𝑖 1 , 𝑌𝑖 0 Observed Outcome
- 36. 𝑌𝑖 1 , 𝑌𝑖 0 Observed Outcome
- 37. 𝑌𝑖 1 , 𝑌𝑖 0 Observed Outcome
- 38. 𝑌𝑖 1 , 𝑌𝑖 0 Observed Outcome
- 39. 𝑌𝑖 1 , 𝑌𝑖 0 Observed Outcome Fundamental Identification Problem of Program Impact Evaluation
- 40. 𝑌𝑖 1 , 𝑌𝑖 0 Observed Outcome Fundamental Identification Problem of Program Impact Evaluation
- 41. Individual Population
- 42. Individual Population Hi. They call me individual i
- 43. Individual Population ?!?
- 44. 𝑌𝑖 1 , 𝑌𝑖 0
- 45. 𝑌𝑖 1 , 𝑌𝑖 0
- 46. An expected value for a random variable is the average value from a large number of repetitions of the experiment that random variable represents An expected value is the true average of a random variable across a population Expected Value
- 47. An expected value for a random variable is the average value from a large number of repetitions of the experiment that random variable represents An expected value is the true average of a random variable across a population Expected Value
- 48. An expected value is the true average of a random variable across a population 𝐸 𝑋 = sometruevalue Expected Value
- 49. 𝐸 𝑐 = 𝑐 𝐸 𝑐 ∙ 𝑊 = 𝑐 ∙ 𝐸 𝑊 𝐸 𝑊 + 𝑍 = 𝐸 𝑊 + 𝐸 𝑍 𝐸 𝑊 − 𝑍 = 𝐸 𝑊 − 𝐸 𝑍 𝐸 𝑎 ∙ 𝑊 ± 𝑏 ∙ 𝑍 = 𝑎 ∙ 𝐸 𝑊 ± 𝑏 ∙ 𝐸 𝑍 Expectations: Properties
- 50. 𝑬 𝒄 = 𝒄 𝐸 𝑐 ∙ 𝑊 = 𝑐 ∙ 𝐸 𝑊 𝐸 𝑊 + 𝑍 = 𝐸 𝑊 + 𝐸 𝑍 𝐸 𝑊 − 𝑍 = 𝐸 𝑊 − 𝐸 𝑍 𝐸 𝑎 ∙ 𝑊 ± 𝑏 ∙ 𝑍 = 𝑎 ∙ 𝐸 𝑊 ± 𝑏 ∙ 𝐸 𝑍 Expectations: Properties
- 51. 𝐸 𝑐 = 𝑐 𝑬 𝒄 ∙ 𝑾 = 𝒄 ∙ 𝑬 𝑾 𝐸 𝑊 + 𝑍 = 𝐸 𝑊 + 𝐸 𝑍 𝐸 𝑊 − 𝑍 = 𝐸 𝑊 − 𝐸 𝑍 𝐸 𝑎 ∙ 𝑊 ± 𝑏 ∙ 𝑍 = 𝑎 ∙ 𝐸 𝑊 ± 𝑏 ∙ 𝐸 𝑍 Expectations: Properties
- 52. 𝐸 𝑐 = 𝑐 𝐸 𝑐 ∙ 𝑊 = 𝑐 ∙ 𝐸 𝑊 𝑬 𝑾 + 𝒁 = 𝑬 𝑾 + 𝑬 𝒁 𝐸 𝑊 − 𝑍 = 𝐸 𝑊 − 𝐸 𝑍 𝐸 𝑎 ∙ 𝑊 ± 𝑏 ∙ 𝑍 = 𝑎 ∙ 𝐸 𝑊 ± 𝑏 ∙ 𝐸 𝑍 Expectations: Properties
- 53. 𝐸 𝑐 = 𝑐 𝐸 𝑐 ∙ 𝑊 = 𝑐 ∙ 𝐸 𝑊 𝐸 𝑊 + 𝑍 = 𝐸 𝑊 + 𝐸 𝑍 𝑬 𝑾 − 𝒁 = 𝑬 𝑾 − 𝑬 𝒁 𝐸 𝑎 ∙ 𝑊 ± 𝑏 ∙ 𝑍 = 𝑎 ∙ 𝐸 𝑊 ± 𝑏 ∙ 𝐸 𝑍 Expectations: Properties
- 54. 𝐸 𝑐 = 𝑐 𝐸 𝑐 ∙ 𝑊 = 𝑐 ∙ 𝐸 𝑊 𝐸 𝑊 + 𝑍 = 𝐸 𝑊 + 𝐸 𝑍 𝐸 𝑊 − 𝑍 = 𝐸 𝑊 − 𝐸 𝑍 𝑬 𝒂 ∙ 𝑾 ± 𝒃 ∙ 𝒁 = 𝒂 ∙ 𝑬 𝑾 ± 𝒃 ∙ 𝑬 𝒁 Expectations: Properties
- 55. 𝐸 𝑊 ∙ 𝑍 ≠ 𝐸 𝑊 ∙ 𝐸 𝑍 𝐸 𝑊 𝑍 ≠ 𝐸 𝑊 𝐸 𝑍 𝐸 𝑓 𝑊 ≠ 𝑓 𝐸 𝑊 Expectations: Properties
- 56. 𝑬 𝑾 ∙ 𝒁 ≠ 𝑬 𝑾 ∙ 𝑬 𝒁 𝐸 𝑊 𝑍 ≠ 𝐸 𝑊 𝐸 𝑍 𝐸 𝑓 𝑊 ≠ 𝑓 𝐸 𝑊 Expectations: Properties
- 57. 𝐸 𝑊 ∙ 𝑍 ≠ 𝐸 𝑊 ∙ 𝐸 𝑍 𝑬 𝑾 𝒁 ≠ 𝑬 𝑾 𝑬 𝒁 𝐸 𝑓 𝑊 ≠ 𝑓 𝐸 𝑊 Expectations: Properties
- 58. 𝐸 𝑊 ∙ 𝑍 ≠ 𝐸 𝑊 ∙ 𝐸 𝑍 𝐸 𝑊 𝑍 ≠ 𝐸 𝑊 𝐸 𝑍 𝑬 𝒇 𝑾 ≠ 𝒇 𝑬 𝑾 Expectations: Properties
- 59. 𝑌𝑖 1 − 𝑌𝑖 0 Average Treatment Effect (ATE) 𝐸 𝑌1 − 𝑌0 Average Effect of Treatment on the Treated (ATT) 𝐸 𝑌1 − 𝑌0|𝑃 = 1 Hi there Individual Impact
- 60. 𝑌𝑖 1 − 𝑌𝑖 0
- 61. 𝐸 𝑌𝑖 1 − 𝑌𝑖 0
- 62. Average Treatment Effect (ATE) 𝐸 𝑌1 − 𝑌0 Average Effect of Treatment on the Treated (ATT) 𝐸 𝑌1 − 𝑌0|𝑃 = 1 Treatment Effects
- 63. Suppose that we have a sample of 𝑖 = 1,…, 𝑛 individuals…. …but for each individual 𝑖 we observe either 𝑌𝑖 1 or 𝑌𝑖 0 … …but not both So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 64. Suppose that we have a sample of 𝑖 = 1,…, 𝑛 individuals…. …but for each individual 𝑖 we observe either 𝑌𝑖 1 or 𝑌𝑖 0 … …but not both So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 65. Remember, however, a key property of expectations: 𝐸 𝑌1 − 𝑌0 = 𝐸 𝑌1 − 𝐸 𝑌0 …but this means that in principle we could estimate E 𝑌1 and E 𝑌0 separately So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 66. Remember, however, a key property of expectations: 𝐸 𝑌1 − 𝑌0 = 𝐸 𝑌1 − 𝐸 𝑌0 …but this means that in principle we could estimate E 𝑌1 and E 𝑌0 separately So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 67. For instance, suppose that in our sample we have: 𝑛 𝑃 participants(𝑃𝑖 = 1) and 𝑛 𝑁 non-participants(𝑃𝑖 = 0) (hence 𝑛 𝑃 + 𝑛 𝑁 = 𝑛) So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 68. Then an estimator of 𝐸 𝑌1 is 𝑌1 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 calculated with the 𝑛 𝑃 participants out of the sample of 𝑛 individuals So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 69. Then an estimator of 𝐸 𝑌1 is 𝒀 𝟏 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 calculated with the 𝑛 𝑃 participants out of the sample of 𝑛 individuals So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 70. Then an estimator of 𝐸 𝑌1 is 𝑌1 = 𝑗=1 𝒏 𝑷 𝑌𝑗 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 calculated with the 𝑛 𝑃 participants out of the sample of 𝑛 individuals So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 71. Then an estimator of 𝐸 𝑌1 is 𝑌1 = 𝑗=1 𝑛 𝑃 𝒀𝒋 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 calculated with the 𝑛 𝑃 participants out of the sample of 𝑛 individuals So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 72. Then an estimator of 𝐸 𝑌1 is 𝑌1 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝒏 𝑷 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 calculated with the 𝑛 𝑃 participants out of the sample of 𝑛 individuals So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 73. Then an estimator of 𝐸 𝑌1 is 𝑌1 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝒀𝒋 𝟏 𝑛 𝑃 calculated with the 𝑛 𝑃 participants out of the sample of 𝑛 individuals So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 74. Similarly, an estimator of 𝐸 𝑌0 is 𝑌0 = 𝑘=1 𝑛 𝑁 𝑌𝑘 𝑛 𝑁 = 𝑘=1 𝑛 𝑁 𝑌𝑘 0 𝑛 𝑁 calculated with the 𝑛 𝑁 non-participants out of the sample of 𝑛 individuals So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 75. So then an estimate of 𝐸 𝑌1 − 𝑌0 = 𝐸 𝑌1 − 𝐸 𝑌0 is 𝑌1 − 𝑌0 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 − 𝑘=1 𝑛 𝑁 𝑌𝑘 𝑛 𝑁 = 𝑘=1 𝑛 𝑁 𝑌𝑘 0 𝑛 𝑁 − 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 So how do we estimate 𝑬 𝒀 𝟏 − 𝒀 𝟎 ??
- 76. But is it a good estimate??
- 77. So we have two samples of size 𝒏 By random chance, between the two samples we almost surely have 1. A different precise mix of individuals 2. A different number of participants (𝑛 𝑃) and non-participants (𝑛 𝑁) 3. Different estimates 𝑌1 and 𝑌0 of 𝐸 𝑌1 and 𝐸 𝑌0 : 𝑌1 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 𝑌0 = 𝑘=1 𝑛 𝑁 𝑌𝑘 𝑛 𝑁 = 𝑘=1 𝑛 𝑁 𝑌𝑘 0 𝑛 𝑁
- 78. So we have two samples of size 𝒏 By random chance, between the two samples we almost surely have 1. A different precise mix of individuals 2. A different number of participants (𝑛 𝑃) and non-participants (𝑛 𝑁) 3. Different estimates 𝑌1 and 𝑌0 of 𝐸 𝑌1 and 𝐸 𝑌0 : 𝑌1 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 𝑌0 = 𝑘=1 𝑛 𝑁 𝑌𝑘 𝑛 𝑁 = 𝑘=1 𝑛 𝑁 𝑌𝑘 0 𝑛 𝑁
- 79. So we have two samples of size 𝒏 By random chance, between the two samples we almost surely have 1. A different precise mix of individuals 2. A different number of participants (𝑛 𝑃) and non-participants (𝑛 𝑁) 3. Different estimates 𝑌1 and 𝑌0 of 𝐸 𝑌1 and 𝐸 𝑌0 : 𝑌1 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 𝑌0 = 𝑘=1 𝑛 𝑁 𝑌𝑘 𝑛 𝑁 = 𝑘=1 𝑛 𝑁 𝑌𝑘 0 𝑛 𝑁
- 80. So we have two samples of size 𝒏 By random chance, between the two samples we almost surely have 1. A different precise mix of individuals 2. A different number of participants (𝑛 𝑃) and non-participants (𝑛 𝑁) 3. Different estimates 𝑌1 and 𝑌0 of 𝐸 𝑌1 and 𝐸 𝑌0 : 𝑌1 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 𝑌0 = 𝑘=1 𝑛 𝑁 𝑌𝑘 𝑛 𝑁 = 𝑘=1 𝑛 𝑁 𝑌𝑘 0 𝑛 𝑁
- 81. So we have two samples of size 𝒏 By random chance, between the two samples we almost surely have 1. A different precise mix of individuals 2. A different number of participants (𝑛 𝑃) and non-participants (𝑛 𝑁) 3. Different estimates 𝑌1 and 𝑌0 of 𝐸 𝑌1 and 𝐸 𝑌0 : 𝑌1 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 = 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 𝑌0 = 𝑘=1 𝑛 𝑁 𝑌𝑘 𝑛 𝑁 = 𝑘=1 𝑛 𝑁 𝑌𝑘 0 𝑛 𝑁
- 82. So we have two samples of size 𝒏 By random chance, between the two samples we almost surely have 1. A different precise mix of individuals 2. A different number of participants (𝑛 𝑃) and non-participants (𝑛 𝑁) 3. Different estimates 𝑌1 and 𝑌0 of 𝐸 𝑌1 and 𝐸 𝑌0 : 𝒀 𝟏 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝒏 𝑷 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑌0 = 𝑘=1 𝑛 𝑁 𝑌𝑘 𝑛 𝑁 = 𝑘=1 𝑛 𝑁 𝑌𝑘 0 𝑛 𝑁 𝒀 𝟏 𝑬 𝒀 𝟏
- 83. 𝒀 𝟏 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝒏 𝑷 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷
- 84. 𝒀 𝟏 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝒏 𝑷 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷
- 85. 𝒀 𝟏 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝒏 𝑷 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏
- 86. 𝒀 𝟏 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝒏 𝑷 = 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏
- 87. 𝑬 𝒀 𝟏 = 𝑬 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 𝑬 𝒀 𝟏 = 𝒏 𝑷 ∙ 𝑬 𝒋=𝟏 𝒏 𝑷 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝒋=𝟏 𝒏 𝑷 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝒏 𝑷 ∙ 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝑬 𝒀𝒋 𝟏 1ST Rule: 𝑬 𝒄 ∙ 𝑿 = 𝒄 ∙ 𝑬 𝑿 𝑬 𝟏 𝒏 𝑷 ∙ 𝑿 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝑿 2nd Rule: 𝑬 𝑿 + 𝒁 = 𝑬 𝑿 + 𝑬 𝒁 𝑬 𝒊=𝟏 𝒏 𝑷 𝑿𝒊 = 𝒊=𝟏 𝒏 𝑷 𝑬 𝑿𝒊
- 88. 𝑬 𝒀 𝟏 = 𝑬 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 𝑬 𝒀 𝟏 = 𝒏 𝑷 ∙ 𝑬 𝒋=𝟏 𝒏 𝑷 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝒋=𝟏 𝒏 𝑷 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝒏 𝑷 ∙ 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝑬 𝒀𝒋 𝟏 1ST Rule: 𝑬 𝒄 ∙ 𝑿 = 𝒄 ∙ 𝑬 𝑿 𝑬 𝟏 𝒏 𝑷 ∙ 𝑿 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝑿 2nd Rule: 𝑬 𝑿 + 𝒁 = 𝑬 𝑿 + 𝑬 𝒁 𝑬 𝒊=𝟏 𝒏 𝑷 𝑿𝒊 = 𝒊=𝟏 𝒏 𝑷 𝑬 𝑿𝒊
- 89. 𝑬 𝒀 𝟏 = 𝑬 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 1ST Rule: 𝑬 𝒄 ∙ 𝑿 = 𝒄 ∙ 𝑬 𝑿 𝑬 𝟏 𝒏 𝑷 ∙ 𝑿 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝑿 1ST Rule: 𝑬 𝒄 ∙ 𝑿 = 𝒄 ∙ 𝑬 𝑿 𝑬 𝟏 𝒏 𝑷 ∙ 𝑿 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝑿 2nd Rule: 𝑬 𝑿 + 𝒁 = 𝑬 𝑿 + 𝑬 𝒁 𝑬 𝒊=𝟏 𝒏 𝑷 𝑿𝒊 = 𝒊=𝟏 𝒏 𝑷 𝑬 𝑿𝒊
- 90. 𝑬 𝒀 𝟏 = 𝑬 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝒋=𝟏 𝒏 𝑷 𝑬 𝒀𝒋 𝟏 1ST Rule: 𝑬 𝒄 ∙ 𝑿 = 𝒄 ∙ 𝑬 𝑿 𝑬 𝟏 𝒏 𝑷 ∙ 𝑿 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝑿 1ST Rule: 𝑬 𝒄 ∙ 𝑿 = 𝒄 ∙ 𝑬 𝑿 𝑬 𝟏 𝒏 𝑷 ∙ 𝑿 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝑿 2nd Rule: 𝑬 𝑿 + 𝒁 = 𝑬 𝑿 + 𝑬 𝒁 𝑬 𝒊=𝟏 𝒏 𝑷 𝑿𝒊 = 𝒊=𝟏 𝒏 𝑷 𝑬 𝑿𝒊
- 91. 𝑬 𝒀 𝟏 = 𝑬 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝒋=𝟏 𝒏 𝑷 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝒏 𝑷 ∙ 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝑬 𝒀𝒋 𝟏 1ST Rule: 𝑬 𝒄 ∙ 𝑿 = 𝒄 ∙ 𝑬 𝑿 𝑬 𝟏 𝒏 𝑷 ∙ 𝑿 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝑿
- 92. 𝑬 𝒀 𝟏 = 𝑬 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝒋=𝟏 𝒏 𝑷 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝒏 𝑷 ∙ 𝑬 𝒀𝒋 𝟏 1ST Rule: 𝑬 𝒄 ∙ 𝑿 = 𝒄 ∙ 𝑬 𝑿 𝑬 𝟏 𝒏 𝑷 ∙ 𝑿 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝑿
- 93. 𝑬 𝒀 𝟏 = 𝑬 𝒋=𝟏 𝒏 𝑷 𝒀𝒋 𝟏 𝒏 𝑷 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝑬 𝒋=𝟏 𝒏 𝑷 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝟏 𝒏 𝑷 ∙ 𝒏 𝑷 ∙ 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝑬 𝒀𝒋 𝟏
- 94. 𝑬 𝒀 𝟏 = 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 𝑬 𝒀𝒋 𝟏 = 𝑬 𝒀 𝟏
- 95. 𝑬 𝒀 𝟏 = 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 𝑬 𝒀𝒋 𝟏 = 𝑬 𝒀 𝟏
- 96. 𝑬 𝒀 𝟏 = 𝑬 𝒀𝒋 𝟏 𝑬 𝒀 𝟏 = 𝑬 𝒀 𝟏 𝑬 𝒀𝒋 𝟏 = 𝑬 𝒀 𝟏
- 97. 𝑬 𝒀𝒋 𝟏 = 𝑬 𝒀 𝟏
- 98. 𝑬 𝒀 𝟏
- 99. 𝑃 = 0 𝑃 = 0 𝑃 = 0 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 0 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 0 𝑃 = 1 𝑃 = 1 𝑃 = 0 𝑃 = 0 𝑃 = 1 𝑃 = 1 𝒀 𝟏
- 100. 𝑃 = 0 𝑃 = 0 𝑃 = 0 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 0 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 0 𝑃 = 1 𝑃 = 1 𝑃 = 0 𝑃 = 0 𝑃 = 1 𝑃 = 0 𝒀 𝟏
- 101. 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝒀 𝟏
- 102. Z W “Z Causes W” 𝑬 𝑾|𝒁 ≠ 𝑬 𝑾
- 103. Z W “Z causes W” 𝑬 𝑾|𝒁 ≠ 𝑬 𝑾
- 104. Z W “Z causes W” 𝑬 𝑾|𝒁 ≠ 𝑬 𝑾
- 105. X Y1
- 106. X Y P
- 107. X Y P 0
- 108. X Y P
- 109. X Y P
- 110. 𝐸 𝑋|𝑃 = 1 ≠ 𝐸 𝑋 𝐸 𝑌1 |𝑃 = 1 ≠ 𝐸 𝑌1 X Y1
- 111. 𝐸 𝑋|𝑃 = 1 ≠ 𝐸 𝑋 𝐸 𝑌1 |𝑃 = 1 ≠ 𝐸 𝑌1 X Y1
- 112. 𝐸 𝑋|𝑃 = 1 ≠ 𝐸 𝑋 𝐸 𝑌1 |𝑃 = 1 ≠ 𝐸 𝑌1 X Y1
- 113. X Y P
- 114. 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1 𝑃 = 1
- 115. 𝐸 𝑌0 |𝑃 = 0 ≠ 𝐸 𝑌0 𝐸 𝑌0 |𝑃 = 1 ≠ 𝐸 𝑌0 𝐸 𝑌0 |𝑃 = 0 ≠ 𝐸 𝑌0 |𝑃 = 0 𝐸 𝑌1 |𝑃 = 0 ≠ 𝐸 𝑌1 𝐸 𝑌1 |𝑃 = 1 ≠ 𝐸 𝑌1 𝐸 𝑌1 |𝑃 = 0 ≠ 𝐸 𝑌1 |𝑃 = 0
- 116. 𝐸 𝑌0 |𝑃 = 0 ≠ 𝐸 𝑌0 𝐸 𝑌0 |𝑃 = 1 ≠ 𝐸 𝑌0 𝐸 𝑌0 |𝑃 = 0 ≠ 𝐸 𝑌0 |𝑃 = 0 𝐸 𝑌1 |𝑃 = 0 ≠ 𝐸 𝑌1 𝐸 𝑌1 |𝑃 = 1 ≠ 𝐸 𝑌1 𝐸 𝑌1 |𝑃 = 0 ≠ 𝐸 𝑌1 |𝑃 = 0
- 117. The estimator 𝑌1 − 𝑌0 = 𝑗=1 𝑛 𝑃 𝑌𝑗 𝑛 𝑃 − 𝑘=1 𝑛 𝑁 𝑌𝑘 𝑛 𝑁 = 𝑘=1 𝑛 𝑁 𝑌𝑘 0 𝑛 𝑁 − 𝑗=1 𝑛 𝑃 𝑌𝑗 1 𝑛 𝑃 of 𝐸 𝑌1 − 𝑌0 would be biased if some individuals occurred only among participants or non-participants Or more often among one of the two groups
- 118. X Y P
- 119. X Y P
- 120. Sir Austin Bradford Hill
- 121. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 122. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 123. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 124. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 125. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 126. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 127. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 128. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 129. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 130. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 131. Strength: How strong is the relationship? Consistency: How consistently is link found? Specificity: How specific is the setting or disease? Temporality: Does the cause precede the effect? Gradient: Does more cause lead to more effect? Analogy: Do similar “causes” have similar effect? Coherence: Are field and laboratory findings similar? Experiment: Was variation in the cause random? Plausibility: Does theory agree? Bradford Hill Criteria
- 132. We are presented with data in the form of a sample: Causality: Our Approach 𝒀𝒊, 𝑷𝒊, 𝑿𝒊 , 𝒊 = 𝟏, . . , 𝒏
- 133. We are presented with data in the form of a sample: Causality: Our Approach 𝒀𝒊, 𝑷𝒊, 𝑿𝒊 , 𝒊 = 𝟏, . . , 𝒏 Assumptions Model E(Y1-Y0), E(Y1-Y0|P=1), Etc.
- 134. We are presented with data in the form of a sample: Causality: Our Approach 𝒀𝒊, 𝑷𝒊, 𝑿𝒊 , 𝒊 = 𝟏, . . , 𝒏 Assumptions Model E(Y1-Y0), E(Y1-Y0|P=1), Etc.
- 135. Conclusion
- 136. Links: The manual: http://www.measureevaluation.org/resources/publications/ms- 14-87-en The webinar introducing the manual: http://www.measureevaluation.org/resources/webinars/metho ds-for-program-impact-evaluation My email: pmlance@email.unc.edu
- 137. MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) under terms of Cooperative Agreement AID-OAA-L-14-00004 and implemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International, John Snow, Inc., Management Sciences for Health, Palladium Group, and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government. www.measureevaluation.org

No public clipboards found for this slide

Be the first to comment