Upcoming SlideShare
Loading in...5







Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

epstein_stata_november_2008.ppt epstein_stata_november_2008.ppt Presentation Transcript

  • Automated stress tests for econometric models Presentation to Stata 2008 Fall North American Users Group Meeting by Roy J. Epstein, PhD Adjunct Professor of Finance, Boston College November 14, 2008
  • Need for Greater Quality Control and Transparency in Applied Econometrics
    • Econometric models now part of mainstream in policy analysis, legal proceedings, and other advocacy situations
    • But assessing reliability of estimation results often difficult
      • Lack of transparency  Risk of bad decisions
    • Enhanced regression software—a virtual referee—is key to improving the practice of econometrics
  • Regression Software as Virtual Referee for Econometric “Best Practices”
    • Journal referee process and professional dialogues routinely probe effects of different:
      • Explanatory variables
      • Data
      • Statistical procedures
    • Similar quality control process can/should be built into regression software
      • Promotes neutral and transparent assessment of reliability
  • Automating Regression Stress Tests
    • Software can anticipate reasonable perturbations of a model
    • Program then can assess sensitivity of original regression to the perturbations
    • Can be standard part of regression output
      • Much as we now expect to see t-statistics
  • Extending Stata for Stress Tests
    • Macro facility allows design of highly flexible testing strategy
    • Can implement as stand-alone post-estimation routine
    • Probably more advantageous to incorporate as part of standard output
    • Can reflect ongoing professional discussion of “best practice” stress tests in econometrics
  • A Taxonomy of Stress Tests
    • Algebraic sign of regression coefficient
    • Magnitude
    • Statistical significance
    • Panel models:
      • Validity of pooling assumptions
      • Cross-sectional error correlation
    • Autocorrelation
    • Temporal aggregation
    • Lagged variables
    • Dropping variables
    • Outliers
    • Functional form
    • Weighting
  • Stata Command (Prototype)
    • stresstest price focuscoef cost realgdp i.firm i.density i.month, timeseries(year month) panel(firm density) w(pounds);
    • Based on “regress” command syntax
    • Parameters identify time, panel, and weighting variables as appropriate
    • Implemented as conventional Stata 10 ado file
  • Case Studies
  • Conwood v. US Tobacco
    • Econometric analysis of tobacco sales
      • Logic of plaintiff model was such that the more negative the estimated slope, the greater the damages.
    • $1 billion claimed damages, after trebling
    • Sustained after review by Supreme Court
    • Timely use of stress tests probably would have vindicated defendant
  • The Conwood Data and Regression Line Washington, DC
  • Stress Test Easily Identifies Effect of Outlier Negative estimated slope, $1 billion damages Positive estimated slope, (excludes Washington, DC) Zero damages
  • Confidential Antitrust Case
    • Alleged price-fixing conspiracy
    • Panel model of average prices (by firm, product, month)
    • Nearly 100 control variables
    • Claimed damages of $330 million based on “statistically significant” coefficient of dummy variable for conspiracy period
  • Why the Virtual Referee Would Have Helped
    • Plaintiff expert:
      • Did not test validity of pooling data for 4 different firms
      • Did not report results from aggregated model but admitted having estimated it
      • Ignored serial correlation in the residuals
      • Did not report effect of excluding control variables with dubious justification
    • Stress tests quickly cut through complexity of model to identify serious reliability problems
  • Stress Tests Reveal High Sensitivity to Pooling Data from Different Firms
  • Stress Tests Reveal High Sensitivity to Aggregating Across the Panel
  • Stress Tests Reveal High Sensitivity to Autocorrelation
  • Stress Tests Reveal High Sensitivity to Presence of Certain Variables
  • Summary
    • Software designers have important role in improving the practice of econometrics
    • Regression results not reliable until sensitivities identified and explained
    • Software can and should automatically invoke appropriate best practice stress tests
    • Software will act as virtual referee
  • For Further Information…
    • Roy J. Epstein, PhD
    • Expert economic analysis for complex litigation
    • 1280 Massachusetts Ave., 2nd Fl.
    • Cambridge, MA 02138
    • [email_address]
    • (617) 489-3818
    • Adjunct Professor of Finance, Boston College