• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content







Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    epstein_stata_november_2008.ppt epstein_stata_november_2008.ppt Presentation Transcript

    • Automated stress tests for econometric models Presentation to Stata 2008 Fall North American Users Group Meeting by Roy J. Epstein, PhD Adjunct Professor of Finance, Boston College November 14, 2008
    • Need for Greater Quality Control and Transparency in Applied Econometrics
      • Econometric models now part of mainstream in policy analysis, legal proceedings, and other advocacy situations
      • But assessing reliability of estimation results often difficult
        • Lack of transparency  Risk of bad decisions
      • Enhanced regression software—a virtual referee—is key to improving the practice of econometrics
    • Regression Software as Virtual Referee for Econometric “Best Practices”
      • Journal referee process and professional dialogues routinely probe effects of different:
        • Explanatory variables
        • Data
        • Statistical procedures
      • Similar quality control process can/should be built into regression software
        • Promotes neutral and transparent assessment of reliability
    • Automating Regression Stress Tests
      • Software can anticipate reasonable perturbations of a model
      • Program then can assess sensitivity of original regression to the perturbations
      • Can be standard part of regression output
        • Much as we now expect to see t-statistics
    • Extending Stata for Stress Tests
      • Macro facility allows design of highly flexible testing strategy
      • Can implement as stand-alone post-estimation routine
      • Probably more advantageous to incorporate as part of standard output
      • Can reflect ongoing professional discussion of “best practice” stress tests in econometrics
    • A Taxonomy of Stress Tests
      • Algebraic sign of regression coefficient
      • Magnitude
      • Statistical significance
      • Panel models:
        • Validity of pooling assumptions
        • Cross-sectional error correlation
      • Autocorrelation
      • Temporal aggregation
      • Lagged variables
      • Dropping variables
      • Outliers
      • Functional form
      • Weighting
    • Stata Command (Prototype)
      • stresstest price focuscoef cost realgdp i.firm i.density i.month, timeseries(year month) panel(firm density) w(pounds);
      • Based on “regress” command syntax
      • Parameters identify time, panel, and weighting variables as appropriate
      • Implemented as conventional Stata 10 ado file
    • Case Studies
    • Conwood v. US Tobacco
      • Econometric analysis of tobacco sales
        • Logic of plaintiff model was such that the more negative the estimated slope, the greater the damages.
      • $1 billion claimed damages, after trebling
      • Sustained after review by Supreme Court
      • Timely use of stress tests probably would have vindicated defendant
    • The Conwood Data and Regression Line Washington, DC
    • Stress Test Easily Identifies Effect of Outlier Negative estimated slope, $1 billion damages Positive estimated slope, (excludes Washington, DC) Zero damages
    • Confidential Antitrust Case
      • Alleged price-fixing conspiracy
      • Panel model of average prices (by firm, product, month)
      • Nearly 100 control variables
      • Claimed damages of $330 million based on “statistically significant” coefficient of dummy variable for conspiracy period
    • Why the Virtual Referee Would Have Helped
      • Plaintiff expert:
        • Did not test validity of pooling data for 4 different firms
        • Did not report results from aggregated model but admitted having estimated it
        • Ignored serial correlation in the residuals
        • Did not report effect of excluding control variables with dubious justification
      • Stress tests quickly cut through complexity of model to identify serious reliability problems
    • Stress Tests Reveal High Sensitivity to Pooling Data from Different Firms
    • Stress Tests Reveal High Sensitivity to Aggregating Across the Panel
    • Stress Tests Reveal High Sensitivity to Autocorrelation
    • Stress Tests Reveal High Sensitivity to Presence of Certain Variables
    • Summary
      • Software designers have important role in improving the practice of econometrics
      • Regression results not reliable until sensitivities identified and explained
      • Software can and should automatically invoke appropriate best practice stress tests
      • Software will act as virtual referee
    • For Further Information…
      • Roy J. Epstein, PhD
      • Expert economic analysis for complex litigation
      • 1280 Massachusetts Ave., 2nd Fl.
      • Cambridge, MA 02138
      • [email_address]
      • (617) 489-3818
      • Adjunct Professor of Finance, Boston College