The document discusses Goodhart's Law in the context of data science, emphasizing that when a measure becomes a target, it loses its effectiveness as a measure. It covers various examples, the importance of cross-validation, tracking model drift, and the necessity of interpretability in models to counteract biases. Data scientists should be aware of these challenges and strive for meaningful metrics and explanatory models to mitigate the effects of Goodhart's Law.

1. Data Science and
Goodhart’s Law
Kyle Polich
Data Science, Inc.

2. Goodhart’s Law
2
When a measure becomes a target, it ceases to be a good measure

3. Sales Rep Compensation Example
• Base pay + variable commission
• For monthly <50k, commission = 3%
• For monthly 50-99k, commission = 5%
• For monthly 100k+, commission = 7%
3

4. Some Examples
 Spam filtering arms race
 Search engine ranking
 Clearing cookies to get better airline prices
 Keep account open to manipulate FICO score
 Retail discounting/couponing strategies
 Bidding in AdTech marketplaces
4

5. Measuring with Cross Validation
Cross Validation
• You should be doing this anyway!
• Set production performance expectation
• Measure post deployment
• Total deviation =
deviation due to overfit
+ deviation due to incomplete training
+ deviation due to Goodhart’s Law
5

6. Measuring via Homogeneity Assumption
Can you train a model to accurately
predict the date at which the observation was created?
6

7. Measuring Drift
7

8. Measuring Drift
8
Typical failure from a web application release

9. Measuring Drift
9
Possible failure from a web application release

10. Dealing with it
• Detection is key
• Experimentation is required
• Agile methods for model deployment
10

11. Causal Impact
• An approach to estimating
the causal effect of a
designed intervention on a
time series.
• Predicts counterfactual
(how response likely would
have evolved absent the
intervention)
11

12. Self Fulfilling Prophecies
• Beware!
• Case study: lead qualification
– Try to predict leads that will close
– Relearn the bias of your training
12

13. Fast Iterations
• Outside normal SWLC release cycle
– State updates
– Parameter tuning
• Run experiments
13

14. Explanatory power
• Goodhart’s law will often manifest on only a
subset of (possibly significant) instances.
• Model interpretability for effected instances is
key
14

15. Interpretable Models
15

16. Interpretable Models
16

17. Why Should I Trust You?
Explaining the Predictions of Any Classifier
Ribeiro, Singh, Guestrin
17
Model Interpretability

18. Summary
• Goodhart’s law: When a measure becomes a target, it ceases to be
a good measure
• As a data scientist, if your work is meaningful, you will encounter it
• Try to measure it in the data
• Work on explanatory models to mitigate
• Don’t let the average case blind you
18

19. DataScience
19
facebook.com/datascience
@DataSkeptic
@datascienceinc
linkedin.com/company/datascience-inc
(310) 579 - 6200