Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Essential economics for data scientists
Benjamin S. Skrainka
February 10, 2016
Benjamin S. Skrainka Essential economics fo...
Overview
Economics studies allocation of resources under scarcity. Many of these
tools are useful for data scientists:
Eco...
Theory I won’t discuss
Economic theory I won’t discuss:
Understanding individual behavior
Understanding firm behavior
Strat...
Applied tools I won’t discuss
Econometric tools I won’t discuss:
Structural vs. reduced form
Bayesian vs. frequentist
Coun...
Objectives
Today’s goals:
List differences between economic & machine learning approach
Know when to use econometrics or ma...
Agenda
Today’s agenda
1 Econometrics or machine learning?
2 Establishing causality
3 When A/B tests fail. . .
4 Causal reg...
References (1/2)
A few references:
Angrist, Joshua D., and Jörn-Steffen Pischke. Mostly harmless
econometrics: An empiricis...
References (2/2)
A few more:
Imbens, Guido W., and Donald B. Rubin. Causal inference in statistics,
social, and biomedical...
Econometrics or machine learning?
Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 9 / 40
Econometrics vs. machine learning
Econometrics Machine learning
Approach statistical: data
generating process
algorithmic ...
Establishing causality
Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 11 / 40
How economists think about data
Data has a data generating process (DGP):
Dependent and independent variables are stochast...
Variation in data
What is the nature of the variation in the data?
Identify exogenous vs. endogenous sources of variation
...
Experimental vs. observational data
In the Rubin Causal Model, experimental data satisfies:
1 Individualistic: whether I am...
Causality and ceteris paribus
Measuring causality depends on ceteris paribus:
Ceteris paribus means “all else being equal”...
Model Interpretation
Interpretation based on:
Partial effects:
Continuous w: βw =
∂E[y|w, c]
∂w
Discrete w: βw = ∆w E[y|w, ...
Establishing Causality
It is very difficult to establish causality:
Provide strong evidence of causality:
Well-designed expe...
Example 1: selection bias (1/2)
Suppose we measure impact of a policy intervention (e.g., advertising), ˆγ:
Yi (Wi ) = µ +...
Example 1: selection bias (2/2)
Selection bias is everywhere in behavioral data:
Randomize to eliminate selection!
In the ...
Endogeneity
Consider a regression model:
yi = xi β + i
If weak exogeneity, E[xi · i ] = 0, holds (among other assumptions)...
Types of endogeneity
There are several types of endogeneity:
Simultaneity
Omitted variable bias (OVB)
Selection bias
Measu...
Common endogenous variables
Endogeneity is everywhere:
Price & demand
Product characteristics
Wages and schooling (or anyt...
Example 2: OLS & endogeneity
Quick review of OLS in one dimension:
yi = β0 + β1 · xi + i
Cov(yi , xi ) = β1 · Var(xi ) + C...
Omitted variable bias (OVB)
OVB is a common problem:
yi = β · xi + α · zi + i
But, zi is omitted from regression
Then, effe...
Individual heterogeneity
Handling individual heterogeneity is a key triumph of modern econometrics:
Unobserved effects from...
Assembling a dataset
Economists think hard about the relationship between features and
outcomes, and what is lurking in th...
Feature engineering à la economics
To handle these problems, economists add supplementary data:
Instrumental variables
Pro...
Natural experiments
Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 28 / 40
Overview
Sometimes A/B testing is not possible:
Impossible to run the experiment
Look for natural randomization devices wh...
A natural experiment
Often ‘nature’ provides natural randomization which is as good as
experimental randomization.
Example...
Causal regression analysis
Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 31 / 40
Causal regression analysis
To establish causality, must condition on other variables so ceteris paribus
applies:
Want to e...
Some common regression tools
Econometricians have developed many methods to assess causal
relationships:
Regression discon...
Regression discontinuity design
Can exploit policy or laws which cause a ‘natural’ treatment effect to
establish causality:...
Difference-in-differences (DID) (1/2)
DID useful when you have individual effects and common time trends:
Must observe data o...
DID regression (2/2)
Can estimate DID using regression:
yit = αi + δt + β · Wi + γ · Wi × PERIODt + it
Wi is treatment sta...
Instrumental variables
An instrumental variable, z, provides a way to correct for endogeneity:
Assumptions:
E[ i · zi ] = ...
Panel data
Panel data is a powerful tool to eliminate sources of bias:
yit = αi + δt + xitβi + it
Panel data consists of i...
Least squares dummy variable regression (LSDV)
Often panel data is equivalent to LSDV:
Occurs when individual effect could ...
Conclusion
To establish a causal relationship:
Must furnish evidence of correctness
Experiment is gold standard
In absence...
Upcoming SlideShare
Loading in …5
×

Essential econometrics for data scientists

1,405 views

Published on

Econometrics or machine learning. I explain which each tool is appropriate, and survey the issues and tools involved in establishing causal relationships.

Published in: Data & Analytics
  • If we are speaking about saving time and money this site ⇒ www.HelpWriting.net ⇐ is going to be the best option!! I personally used lots of times and remain highly satisfied.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THE BOOK INTO AVAILABLE FORMAT (New Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THE can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THE is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBOOK .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookBOOK, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, EBOOK, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THE Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THE the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THE Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • The Insider's Edge You've Been Looking For.... ★★★ http://t.cn/A6hP86vM
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I have barely snored at all! My girlfriend was starting to put pressure on me to have an operation to stop my snoring, but to be totally honest I was scared stiff. I've heard some horror stories and there was no way I wanted to take on that risk. Then I found your website and since putting your techniques into practice I have barely snored at all. My girlfriend can't believe how effective this has been. ▲▲▲ http://ishbv.com/snoringno/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • The Odds of The Lotto Are Changing With These Tricks! ▲▲▲ https://tinyurl.com/t2onem4
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Essential econometrics for data scientists

  1. 1. Essential economics for data scientists Benjamin S. Skrainka February 10, 2016 Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 1 / 40
  2. 2. Overview Economics studies allocation of resources under scarcity. Many of these tools are useful for data scientists: Econometric methods adapt classical statistics for applied problems Causal inference Experimental design Regression analysis Often, require small or ‘medium’ data Goal of talk: understand (magnitude) of causal relationships Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 2 / 40
  3. 3. Theory I won’t discuss Economic theory I won’t discuss: Understanding individual behavior Understanding firm behavior Strategic questions: products, pricing, auctions, platforms, incentives, M&A, new products Estimate demand & forecasting Structural modeling Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 3 / 40
  4. 4. Applied tools I won’t discuss Econometric tools I won’t discuss: Structural vs. reduced form Bayesian vs. frequentist Counter-factual & welfare analysis Forecasting Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 4 / 40
  5. 5. Objectives Today’s goals: List differences between economic & machine learning approach Know when to use econometrics or machine learning Survey alternative types of experiments Overview of how to estimate causal effects using regression analysis Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 5 / 40
  6. 6. Agenda Today’s agenda 1 Econometrics or machine learning? 2 Establishing causality 3 When A/B tests fail. . . 4 Causal regression analysis Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 6 / 40
  7. 7. References (1/2) A few references: Angrist, Joshua D., and Jörn-Steffen Pischke. Mostly harmless econometrics: An empiricist’s companion. Princeton university press, 2008. Angrist, Joshua D., and Jörn-Steffen Pischke. Mastering ’metrics: The path from cause to effect. Princeton University Press, 2014. Breiman, Leo. “Statistical modeling: The two cultures (with comments and a rejoinder by the author).” Statistical Science 16.3 (2001): 199-231. Cameron, A. Colin, and Pravin K. Trivedi. Microeconometrics: methods and applications. Cambridge university press, 2005. Card, David, and Alan B. Krueger. “Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania.” The American Economic Review 84.4 (1994): 772-793. Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 7 / 40
  8. 8. References (2/2) A few more: Imbens, Guido W., and Donald B. Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015. LaLonde, Robert J. “Evaluating the econometric evaluations of training programs with experimental data.” The American economic review (1986): 604-620. Pearl, Judea. Causality. Cambridge university press, 2009. Wooldridge, Jeffrey M. Econometric analysis of cross section and panel data. MIT press, 2010. Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 8 / 40
  9. 9. Econometrics or machine learning? Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 9 / 40
  10. 10. Econometrics vs. machine learning Econometrics Machine learning Approach statistical: data generating process algorithmic model, DGP unknown Driver theory fitting the data Focus hypothesis testing & interpretability predictive accuracy Model choice parameter significance & in-sample goodness of fit cross-validation of predictive accuracy on partitions of data Strength understand causal relationships & behavior prediction See Breiman (2001) and Matt Bogard’s blog Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 10 / 40
  11. 11. Establishing causality Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 11 / 40
  12. 12. How economists think about data Data has a data generating process (DGP): Dependent and independent variables are stochastic A structure is the statistical & functional relationship that determines the observed outcomes Two structures are observationally equivalent if they produce the same process A structure is identified only if a unique structure can cause the process Consequently: Parameter estimates are random Can perform inference on them. . . . . . if the inverse problem is well-posed, i.e., the model is identified Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 12 / 40
  13. 13. Variation in data What is the nature of the variation in the data? Identify exogenous vs. endogenous sources of variation Exogenous: Variable is determined outside the model Example: cost, weather, draft lottery number, parental income Endogenous: Variable is determined inside the model Caused by E[x · ] = 0 or causal loop between y and x Example: crime & policing; price & demand, product characteristics ⇒ Mishandling endogenous features/variables almost always causes biased estimates Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 13 / 40
  14. 14. Experimental vs. observational data In the Rubin Causal Model, experimental data satisfies: 1 Individualistic: whether I am assigned to treatment doesn’t affect whether you are 2 Probabilistic: non-zero probability of assignment to each treatment 3 Unconfoundedness: outcome doesn’t affect probability of assignment 4 Known, random assignment rule If 4. is violated, your data is observational Also need Stable Unit Treatment Value Assumption (SUTVA) Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 14 / 40
  15. 15. Causality and ceteris paribus Measuring causality depends on ceteris paribus: Ceteris paribus means “all else being equal” I.e., compare apples to apples by conditioning on everything other than the variable under analysis E.g., data should be as good as randomly assigned In the terminology of Wooldridge (2010): Want to understand how w affects y Must condition on other confounding influences or correlation between w and c will bias results: E[y|w, c] Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 15 / 40
  16. 16. Model Interpretation Interpretation based on: Partial effects: Continuous w: βw = ∂E[y|w, c] ∂w Discrete w: βw = ∆w E[y|w, c] Elasticities: η = w · ∂E[y|w, c] E[y|w, c] · ∂w = ∂ log E[y|w, c] ∂ log w Captures dimensionless change E.g., market power: price − marginal cost price = − 1 ηDemand Structural models permit more sophisticated counter-factual analysis Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 16 / 40
  17. 17. Establishing Causality It is very difficult to establish causality: Provide strong evidence of causality: Well-designed experimentation Careful statistical analysis Show method controls for sources of possible bias Randomization is the gold standard For observational data, must show controls make (regression) analysis ‘as good as randomly assigned’ See LaLonde (1986) or Angrist & Pischke (2008, 2014) Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 17 / 40
  18. 18. Example 1: selection bias (1/2) Suppose we measure impact of a policy intervention (e.g., advertising), ˆγ: Yi (Wi ) = µ + γ · Wi + i Try differencing averages, but we only observe outcomes condition on treatment status Wi : observed effect = AVGn[Yi (1)|Wi = 1] − AVGn[Yi (0)|Wi = 0] But: observed effect = AVGn[Yi (1)|Wi = 1] − AVGn[Yi (1)|Wi = 0] direct effect + AVGn[Yi (1)|Wi = 0] − AVGn[Yi (0)|Wi = 0] selection Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 18 / 40
  19. 19. Example 1: selection bias (2/2) Selection bias is everywhere in behavioral data: Randomize to eliminate selection! In the absence of randomization: Model selection process Choose sensible functional form & distributions Use control functional Condition on suitable controls to compare groups which are as good as randomly assigned See Wooldridge (2010) Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 19 / 40
  20. 20. Endogeneity Consider a regression model: yi = xi β + i If weak exogeneity, E[xi · i ] = 0, holds (among other assumptions) ⇒ ˆβ unbiased Endogeneity occurs if yi and xi are codetermined: Weak exogeneity fails, E[xi · i ] = 0 ˆβ is biased Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 20 / 40
  21. 21. Types of endogeneity There are several types of endogeneity: Simultaneity Omitted variable bias (OVB) Selection bias Measurement error If any are present, E[xi · i ] = 0 and your estimates are biased Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 21 / 40
  22. 22. Common endogenous variables Endogeneity is everywhere: Price & demand Product characteristics Wages and schooling (or anything affected by ability) Labor-force participation Policing & crime Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 22 / 40
  23. 23. Example 2: OLS & endogeneity Quick review of OLS in one dimension: yi = β0 + β1 · xi + i Cov(yi , xi ) = β1 · Var(xi ) + Cov( i , xi ) ˆβ1 = β1 + Cov( i , xi ) Var(xi ) ⇒ ˆβ1 is unbiased only ⇔ Cov( i , xi ) = 0 Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 23 / 40
  24. 24. Omitted variable bias (OVB) OVB is a common problem: yi = β · xi + α · zi + i But, zi is omitted from regression Then, effective error term is ui = i + α · zi and E[ui , xi ] = 0 Estimates are biased: ˆβ = Cov(yi , xi ) Var(xi ) + α · Cov(zi , xi ) Var(xi ) Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 24 / 40
  25. 25. Individual heterogeneity Handling individual heterogeneity is a key triumph of modern econometrics: Unobserved effects from individuals and firms affect behavior Failure to model, can cause biased or inefficient estimates Often, can use panel data models to control for an unobserved effect: Panel data is a data where we observed individuals over time Can exploit this to eliminate temporal and individual biases: yit = αi + δt + xitβi + uit Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 25 / 40
  26. 26. Assembling a dataset Economists think hard about the relationship between features and outcomes, and what is lurking in the error term: Think about the error term: Endogeneity? Omitted variables? Valid instruments? Seek out supplementary data which could explain behavior . . . or proxy for missing features Choose a functional form to control for ignorance yet measure what matters Create a panel to control for individual heterogeneity Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 26 / 40
  27. 27. Feature engineering à la economics To handle these problems, economists add supplementary data: Instrumental variables Proxy variables dummy variables for individual Or, use clever tricks: Panel data Regression discontinuity design Difference-in-differences Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 27 / 40
  28. 28. Natural experiments Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 28 / 40
  29. 29. Overview Sometimes A/B testing is not possible: Impossible to run the experiment Look for natural randomization devices which provide as good ‘as random assignment to treatment’: Birthdays Draft lottery numbers Access decisions Natural randomization can eliminate selection Always check for balance! Requires some cleverness. . . Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 29 / 40
  30. 30. A natural experiment Often ‘nature’ provides natural randomization which is as good as experimental randomization. Example: needed to measure lift of experiential marketing campaign: * No experimental design * Ten treatment units * Matching estimators failed * But, short list had 50 sites. . . * Assume (and test) if access is as good as random ⇒ have valid treatment & control groups! Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 30 / 40
  31. 31. Causal regression analysis Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 31 / 40
  32. 32. Causal regression analysis To establish causality, must condition on other variables so ceteris paribus applies: Want to estimate E[y|w, c], where: w is the factor of interest c are other factors, correlated with w which could confound analysis Need good measures of y, w, and c Beware of variables determined by equilibrium: Must deal with simultaneous equation modeling Must avoid endogenous variables E.g., crime vs. policing Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 32 / 40
  33. 33. Some common regression tools Econometricians have developed many methods to assess causal relationships: Regression discontinuity design (RDD): Difference-in-differences (DID): Instrumental variables: use exogenous variables to instrument an endogenous variable Panel data: Other methods include: Matching estimators Censoring & truncation Discrete choice Discrete/continuous choice Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 33 / 40
  34. 34. Regression discontinuity design Can exploit policy or laws which cause a ‘natural’ treatment effect to establish causality: Example: impact of drinking age on mortality People on either side of 21 get different treatment but are essentially identical Wi is a function of age Wi is a discontinuous function of age age is known as the running variable mortalityi = α0 + α1 · age + γ · Wi (age) + i See Agrist & Pischke (2015) Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 34 / 40
  35. 35. Difference-in-differences (DID) (1/2) DID useful when you have individual effects and common time trends: Must observe data over at least two periods Eliminate bias from individual and time effects E.g., impact of minimum wage on employment; See, Card & Krueger (1998) yit = αi + δt + γ · Wi × PERIODt ˆγ = (yNJ,1 − yNJ,0) − (yPA,1 − yPA,0) ˆγ = (∆δt + γ) − (∆δt) Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 35 / 40
  36. 36. DID regression (2/2) Can estimate DID using regression: yit = αi + δt + β · Wi + γ · Wi × PERIODt + it Wi is treatment status PERIODt ∈ 0, 1 for periods 0 and 1, respectively γ is the treatment effect Add additional covariates to control as needed Verify common trends holds (e.g., plot data vs. time) Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 36 / 40
  37. 37. Instrumental variables An instrumental variable, z, provides a way to correct for endogeneity: Assumptions: E[ i · zi ] = 0 E[xi · zi ] = 0 Use z in regression: Cov(yi , zi ) = β1 · Cov(xi , zi ) + Cov( i , zi ) ˆβ1 IV = β1 + Cov( i , zi ) Cov(xi , zi ) Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 37 / 40
  38. 38. Panel data Panel data is a powerful tool to eliminate sources of bias: yit = αi + δt + xitβi + it Panel data consists of individuals observed overtime, i.e., xit Has time series and cross-section properties Can eliminate individual & time effects: With-in estimator ( .. xit ← xit − xi ) First differences (FD) Can also handle serial correlation of { t} Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 38 / 40
  39. 39. Least squares dummy variable regression (LSDV) Often panel data is equivalent to LSDV: Occurs when individual effect could be modeled using dummy variables for each individual Frisch-Waugh decomposition: Powerful dimension reduction Use when you need to control for unobserved effect without estimating it Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 39 / 40
  40. 40. Conclusion To establish a causal relationship: Must furnish evidence of correctness Experiment is gold standard In absence of random assignment to treatment: Natural experiments can provide as good as random assignment to treatment Regression analysis can be causal if you condition on confounding influences and control for endogeneity Benjamin S. Skrainka Essential economics for data scientists February 10, 2016 40 / 40

×