Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- M&E for Social Service System Stren... by MEASURE Evaluation 1674 views
- Fundamentals of Program Impact Eval... by MEASURE Evaluation 2465 views
- Measuring Impact Qualitatively by MEASURE Evaluation 3168 views
- Project Management Cycle: Tackling ... by MEASURE Evaluation 2324 views
- Randomization and Its Discontents by MEASURE Evaluation 889 views
- Impact evaluation by Carlo Magno 848 views

2,734 views

Published on

Presented by Peter M. Lance in a September 2014 webinar.

Published in:
Education

No Downloads

Total views

2,734

On SlideShare

0

From Embeds

0

Number of Embeds

3

Shares

0

Downloads

224

Comments

0

Likes

2

No embeds

No notes for slide

- 1. An Introduction to the Manual: How Do We Know if a Program Made a Difference? A Guide to Statistical Methods for Program Impact Evaluation Peter M. Lance MEASURE Evaluation September 25, 2014
- 2. Where it all began….
- 3. Approachable But Basic Sophisticated But Unapproachable
- 4. Simple but Limited 1. Heavy reliance on verbal explanations -Does not convey much nuance -Does not provide much insight about data structure -Do not typically relate different IE methods well 2. Little guidance on how actually to estimate the models 3. Tend to focus on basics, with little recent thinking
- 5. Simple but Limited 1. Heavy reliance on verbal explanations -Does not convey much nuance -Does not provide much insight about data structure -Do not typically relate different IE methods well 2. Little guidance on how actually to estimate the models 3. Tend to focus on basics, with little recent thinking
- 6. Simple but Limited 1. Heavy reliance on verbal explanations -Does not convey much nuance -Does not provide much insight about data structure -Do not typically relate different IE methods well 2. Little guidance on how actually to estimate the models 3. Tend to focus on basics, with little recent thinking
- 7. Simple but Limited 1. Heavy reliance on verbal explanations -Does not convey much nuance -Does not provide much insight about data structure -Do not typically relate different IE methods well 2. Little guidance on how actually to estimate the models 3. Tend to focus on basics, with little recent thinking
- 8. Simple but Limited 1. Heavy reliance on verbal explanations -Does not convey much nuance -Does not provide much insight about data structure -Do not typically relate different IE methods well 2. Little guidance on how actually to estimate the models 3. Tend to focus on basics, with little recent thinking
- 9. Simple but Limited 1. Heavy reliance on verbal explanations -Does not convey much nuance -Does not provide much insight about data structure -Do not typically relate different IE methods well 2. Little guidance on how actually to estimate the models 3. Tend to focus on basics, with little recent thinking
- 10. Simple but Limited 1. Heavy reliance on verbal explanations -Does not convey much nuance -Does not provide much insight about data structure -Do not typically relate different IE methods well 2. Little guidance on how actually to estimate the models 3. Tend to focus on basics, with little recent thinking
- 11. Sophisticated and Cutting Edge, but… 1. Reliance on dense mathematics -Assume a lot of background knowledge -A lot of arcane, inconsistent notation -Little guidance on the mathematical journey 2. Still provide little practical illustration of how actually to estimate models or practical performance 3. Often Lopsided
- 12. Sophisticated and Cutting Edge, but… 1. Reliance on dense mathematics -Assume a lot of background knowledge -A lot of arcane, inconsistent notation -Little guidance on the mathematical journey 2. Still provide little practical illustration of how actually to estimate models or practical performance 3. Often Lopsided
- 13. Sophisticated and Cutting Edge, but… 1. Reliance on dense mathematics -Assume a lot of background knowledge -A lot of arcane, inconsistent notation -Little guidance on the mathematical journey 2. Still provide little practical illustration of how actually to estimate models or practical performance 3. Often Lopsided
- 14. Sophisticated and Cutting Edge, but… 1. Reliance on dense mathematics -Assume a lot of background knowledge -A lot of arcane, inconsistent notation -Little guidance on the mathematical journey 2. Still provide little practical illustration of how actually to estimate models or practical performance 3. Often Lopsided
- 15. A Tale of Two Manuals: Manual 1 Manual 2 푌1 푌1 The Outcome if Individual i participates 푃 푇 Program Participation/Treatment Exposure of individual i 푇 푡 Time 푁 푛 Number of observations 푃 푁 Size of the population Δ푥 푥 Change in 푥
- 16. Sophisticated and Cutting Edge, but… 1. Reliance on dense mathematics -Assume a lot of background knowledge -A lot of arcane, inconsistent notation -Little guidance on the mathematical journey 2. Still provide little practical illustration of how actually to estimate models or practical performance 3. Often Lopsided
- 17. Obviously leads to 풏≈ 풁ퟏ−휶 ퟐ ퟐ ∙풑∙ퟏ−풑 풅ퟐ ퟏ+ 풁ퟏ−휶 ퟐ ퟐ ∙풑∙ퟏ−풑 풅ퟐ 푵 Obviously…. Totally 푃푟−푑≤푝−푃≤푑|푁≥1−훼 The Inequality
- 18. A B C D E F G
- 19. A B C D E F G
- 20. Sophisticated and Cutting Edge, but… 1. Reliance on dense mathematics -Assume a lot of background knowledge -A lot of arcane, inconsistent notation -Little guidance on the mathematical journey 2. Still provide little practical illustration of how actually to estimate models or practical performance 3. Often Lopsided
- 21. Sophisticated and Cutting Edge, but… 1. Reliance on dense mathematics -Assume a lot of background knowledge -A lot of arcane, inconsistent notation -Little guidance on the mathematical journey 2. Still provide little practical illustration of how actually to estimate models or practical performance 3. Often Lopsided
- 22. Key Guiding Features: 1. Consistent Notation 2. Little Prior Knowledge Assumed 3. Explicit Derivation of All Results in the Manual with verbal/narrative explanation 4. Reliance on a single “behavioral model” woven through the discussion 4. Explicit discussion of the practical estimation
- 23. 5. Equal coverage of major IE traditions 6. Practical examples of how to estimate the IE models and their applied performance in STATA and
- 24. 5. Equal coverage of major IE traditions 6. Practical examples of how to estimate the IE models and their applied performance in STATA and …. All Statistical Programs for them Released with Manual!!!
- 25. Topics Covered 1. Introduction to the basic challenge of IE 2. RCTs 3. Selection on Observables 4. “Within” Estimators 5. Instrumental Variables (including Regression Discontinuity)
- 26. http://www.measureevaluation.org/publications/ms-14-87-en
- 27. Homer Simpson
- 28. Homer Simpson homer@email.deepthinkingu.edu
- 29. Homer Simpson homer@email.deepthinkingu.edu The University of Deep Thinking
- 30. Homer Simpson homer@email.deepthinkingu.edu The University of Deep Thinking Dear Authors, Wow this manual is boring. Too much blah blah blah and not enough cartoons. Yours, Homer
- 31. Homer Simpson homer@email.deepthinkingu.edu The University of Deep Thinking Dear Authors, Wow this manual is boring. Too much blah blah blah and not enough cartoons. Yours, Homer
- 32. The Future 1. Continued revisions to the first edition of the first volume 2. A second edition of the first volume: -Expanded Empirical Examples (adding in new STATA treatment effects commands, examples in R, etc.); -Separate chapter for Regression Discontinuity; -Extending behavioral model to RCTs; -What else?
- 33. The Future 3. A Potential Second Volume -Hybrid models; -Deeply structural models; -Sample selection/control function rooted models; -Aggregate data (e.g. time series basics, synthetic cohort models, etc.); -What else?
- 34. MEASURE Evaluation is funded by the U.S. Agency for International Development (USAID) and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group International, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide.

No public clipboards found for this slide

Be the first to comment