Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

C Star Analytic Presentation

415 views

Published on

Nathan Danneman (L-3 Data Tactics) -- cstar: A New Way to Communicate Importance.

  • Be the first to comment

  • Be the first to like this

C Star Analytic Presentation

  1. 1. Nathan H. Danneman c* -- A new way to communicate importance Esarey and Danneman (forthcoming). A Quantitative Method for Substantive Robustness Assessment. Political Science Research and Methods.
  2. 2. The Point: 1. p-values don’t matter 2. Substantive significance DOES matter 3. Substantive significance is subjective 4. Formalizing substantive significance can help us communicate, and make decisions from evidence
  3. 3. But…in school we learned p-values are really really important!!!
  4. 4. Suppose my friend invented a magic weight loss pill...
  5. 5. ...but, it *might* cause cancer:
  6. 6. How did you choose whether or not to take the drug? Ostensibly, you weighed: 1. The evidence that the drug causes cancer 2. How bad it would be to get cancer 3. The evidence that the drug is safe 4. How good it would be to lose weight
  7. 7. How did you choose whether or not to take the drug? Ostensibly, you weighed: 1. The evidence that the drug causes cancer 2. How bad it would be to get cancer 3. The evidence that the drug is safe 4. How good it would be to lose weight Letting r denote the empirical change in cancer probability, you solved this:
  8. 8. Some of the inputs to your choice are subjective: The evidence that the drug causes cancer -objective How bad it would be to get cancer -subjective The evidence that the drug is safe -objective How good it would be to lose weight -subjective
  9. 9. Implication It is totally unsatisfactory to say things like: “As civil war becomes more prevalent in the state’ s proximity, threatening the state with externalities that could make civil conflict more likely, the state increases its level of repression. Substantively, we find that moving from a conflict- free neighborhood (e.g., Switzerland in 2000) to one riddled with civil conflict (e.g., Swaziland in 1982) generates a change of four-tenths of a [repression measure]-unit.” Danneman and Ritter (2014).Contagious Rebellion and Preemptive Repression
  10. 10. Two problems Problem 1: the utility acting on a false positive might be different than the utility of acting on a false negative. Problem 2: the relative utility of these decisions may vary by person (that whole “subjective thing)
  11. 11. Solution: Part 1 Pick a utility function that can capture the relative weighting the u(false positive) versus u(false negative). Our preferred choice: a kinked, linear utility function: Utility=0
  12. 12. Solution: Part 2 The amount of “kink” or “bend” in the utility function is governed by a single parameter. Let’s call it gamma, such that it’s square root represents the number of times a person values false positives versus false negatives.
  13. 13. Step 3: Plot gamma against EU
  14. 14. That assumes you are willing to get off the couch for E[u]=0 If the minimum amount of benefit you are willing to get off the couch for is, say, c, then we can plot gamma against c* that solves E[u] - c = 0. Too much work? See our R package, {cstar} on CRAN.
  15. 15. Thanks. Nathan Danneman @ndanneman ndanneman@gmail.com Esarey and Danneman (forthcoming). A Quantitative Method for Substantive Robustness Assessment. Political Science Research and Methods.

×