Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Quantitative
Methods
for
Lawyers
Power Laws, Hypothesis
Testing & Statistical
Significance
Class #11
@ computational
comput...
Power Law Distribution
(Scale Free)
This is a Classic and Very Important Distribution
A power law is a special kind of mat...
Power Law Distribution
(Scale Free)
Pareto distribution ( Wealth Distribution )
Zipf's law ( Natural Language Frequency )
...
Power Laws Appear to be a
Common Feature of Legal Systems
Katz, et al (2011)
American Legal Academy
Katz & Stafford (2010)...
Rare Events, Criticality
Power Laws
Rare Events
Criticality
Disorder
Induction
“ [T]here are known knowns; there are things we know we know.
We also know there are known unknowns; that is to say we kno...
Unknown, Unknowns and
Inductivist Reasoning
Philosophy of Science =
How do we Know What We Know?
Black Swan Problem
Even I...
Learning by Falsification
Science Advances Incrementally as Hypotheses
are Falsified
Popperian Perspective
Karl Popper Rejec...
Learning by Falsification
the sun has risen every day for as long as anyone can
remember.
what is the rational proof that i...
Learning by Falsification
No Need to Reject the Hypothesis of Sun Rising
Popper Solution to the Question:
Cannot Really For...
Hypothesis Testing
& Statistical Significance
The Null and
Alternative Hypothesis
Criminal Trial Burden of Proof
Example from Criminal Law:
Must Be Overruled Beyond a R...
The Null and
Alternative Hypothesis
Study is Typically Designed to Determine Whether
a Particular Hypothesis is Supported
...
Example: Coin Flip
Nostradamus
Predicting Coin Flips -
Does you Friend Have the General Ability to Actually
Predict Coin F...
Ho: Cannot Actually Predict Coin Flips
Example: Coin Flip
Nostradamus
H1: Can Actually Predict Coin Flip
(i.e. do so at a ...
Reject the Null versus
Failing to Reject the Null
If We Fail to Reject the Null, we are left with the
assumption of no rel...
Coin Flip Nostradamus:
Binomial Distribution
Here is the Formula for a binomial experiment consisting of
n trials and resu...
4!
Coin Flip Nostradamus:
Binomial Distribution
(3! (4-3)! )(.53
) (.54-3
)
(.125) (.5)
24
( 6(1)
) = .25
Here is the Prob...
4!
Coin Flip Nostradamus:
Binomial Distribution
(3! (4-3)! )(.53
) (.54-3
)
(.125) (.5)
24
( 6(1) ) = .25
Here is the Prob...
Namely, there is a 31.25% Probability that by
Chance he/she would be able to predict at least
3 out of 4
If Our Would Be C...
How Much Do We Need to Be Convinced that Our
Friend is Actually Coin Flip Nostradamus?
Now We Can Calculate Probability As...
Type I v. Type II Error
Type I
v.
Type II
Error
Type I v. Type II Error
It is Depends Upon the Application
Typical Convention is that a 5% Chance of Error is
Acceptable f...
Back To
Coin Flip Nostradamus
Predicts 43 out of 75 Correct
Okay let say Our Coin Flip Nostradamus agrees to run
75 coins ...
Binomial Probability Calculator
http://stattrek.com/tables/binomial.aspx
Binomial Probability Calculator
http://stattrek.com/tables/binomial.aspx
Enter
These
Three
Values
+
Hit Calculate
Binomial Probability Calculator
http://stattrek.com/tables/binomial.aspx
And
These are
the Results
Our P
value
Here is
12....
Coin Flip Nostradamus
In this Case, the P Value is
Our P Value is the Probability of Observing this Data
Given the Null (i...
One Tailed -or-
Two Tailed Tests
In the Coin Flip Nostradamus Example it would be
amazing if our friend could actually fai...
One Tailed -or-
Two Tailed Tests
Stricter Crime Law and the Crime Rate
We are Often Interested in a Non-
Directional Hypot...
One Tailed -or-
Two Tailed Tests
Two Tailed Test
One Tailed Test
(negative direction)
One Tailed Test
(Positive direction)
https://onlinecourses.science.psu.edu/stat500/book/export/html/43
An Example of a
Hypothesis Test
Note: π is Prob
α is the...
https://onlinecourses.science.psu.edu/stat500/book/export/html/43
An Example of a
Hypothesis Test
Want to Make Sure
Sample...
https://onlinecourses.science.psu.edu/stat500/book/export/html/43
An Example of a
Hypothesis Test
Want to Make Sure
Sample...
An Example of a
Hypothesis Test
https://onlinecourses.science.psu.edu/stat500/book/export/html/43
z = (p - P) / σ
where p ...
An Example of a
Hypothesis Test
https://onlinecourses.science.psu.edu/stat500/book/export/html/43
I roll a single die 1,000 times and
obtain a "6" on 204 rolls.
Is there significant evidence to
suggest that the die is not...
Another Example Question
Daniel Martin Katz
@ computational
computationallegalstudies.com
lexpredict.com
danielmartinkatz.com
illinois tech - chica...
Upcoming SlideShare
Loading in …5
×

Quantitative Methods for Lawyers - Class #11 - Power Laws, Hypothesis Testing & Statistical Significance - Professor Daniel Martin Katz

1,393 views

Published on

Quantitative Methods for Lawyers - Class #11 - Power Laws, Hypothesis Testing & Statistical Significance - Professor Daniel Martin Katz

Published in: Law
  • Be the first to comment

  • Be the first to like this

Quantitative Methods for Lawyers - Class #11 - Power Laws, Hypothesis Testing & Statistical Significance - Professor Daniel Martin Katz

  1. 1. Quantitative Methods for Lawyers Power Laws, Hypothesis Testing & Statistical Significance Class #11 @ computational computationallegalstudies.com professor daniel martin katz danielmartinkatz.com lexpredict.com slideshare.net/DanielKatz
  2. 2. Power Law Distribution (Scale Free) This is a Classic and Very Important Distribution A power law is a special kind of mathematical relationship between two quantities. When the frequency of an event varies as a power of some attribute of that event (e.g. its size), the frequency is said to follow a power law.
  3. 3. Power Law Distribution (Scale Free) Pareto distribution ( Wealth Distribution ) Zipf's law ( Natural Language Frequency ) Links on the Internet Citations Richardson's Law for the severity of violent conflicts (wars and terrorism) Population of cities Etc. Examples:
  4. 4. Power Laws Appear to be a Common Feature of Legal Systems Katz, et al (2011) American Legal Academy Katz & Stafford (2010) American Federal Judges Geist (2009) Austrian Supreme Court Smith (2007) U.S. Supreme Court Smith (2007) U.S. Law Reviews Post & Eisen (2000) NY Ct of Appeals
  5. 5. Rare Events, Criticality Power Laws Rare Events Criticality Disorder Induction
  6. 6. “ [T]here are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – there are things we do not know we don't know. ” United States Secretary of Defense Donald Rumsfeld
  7. 7. Unknown, Unknowns and Inductivist Reasoning Philosophy of Science = How do we Know What We Know? Black Swan Problem Even If We Observe White Swan after White Swan cannot induce that all swans are white
  8. 8. Learning by Falsification Science Advances Incrementally as Hypotheses are Falsified Popperian Perspective Karl Popper Rejected Inductivist Reasoning
  9. 9. Learning by Falsification the sun has risen every day for as long as anyone can remember. what is the rational proof that it will rise tomorrow? How can one rationally prove that past events will continue to repeat in the future, just because they have repeated in the past? Of Course, Certain Hypothesis cannot likely be falsified on a Reasonable Time Scale The problem of induction:
  10. 10. Learning by Falsification No Need to Reject the Hypothesis of Sun Rising Popper Solution to the Question: Cannot Really Formulate a Theory that Can Prove that the Sun Will Always Rise Can Develop a Theory that It Rise which will be falsified if the sun fails to rise
  11. 11. Hypothesis Testing & Statistical Significance
  12. 12. The Null and Alternative Hypothesis Criminal Trial Burden of Proof Example from Criminal Law: Must Be Overruled Beyond a Reasonable Doubt Presumption of Innocence Not Possible to Conclusively Prove a Lack of Innocence (with zero doubt)
  13. 13. The Null and Alternative Hypothesis Study is Typically Designed to Determine Whether a Particular Hypothesis is Supported Switch Now To a Scientific Inquiry: Start with Presumption that Hypothesis is Not True (Null Hypothesis) Researcher Must Demonstrate That The Presumption is Unlikely to Be True given the Population
  14. 14. Example: Coin Flip Nostradamus Predicting Coin Flips - Does you Friend Have the General Ability to Actually Predict Coin Flips? How Would You Evaluate This Proposition? How Many Predictions Would Your Friend Have to Get Right For You To Believe They Actually Have Real Ability?
  15. 15. Ho: Cannot Actually Predict Coin Flips Example: Coin Flip Nostradamus H1: Can Actually Predict Coin Flip (i.e. do so at a rate greater than chance) Ho is the Null Hypothesis H1 is the Alternative Hypothesis
  16. 16. Reject the Null versus Failing to Reject the Null If We Fail to Reject the Null, we are left with the assumption of no relationship In the Coin Flip Example, We might have enough evidence to reject the null Remember the default (null) is that there is no relationship Although a Relationship might actually exist
  17. 17. Coin Flip Nostradamus: Binomial Distribution Here is the Formula for a binomial experiment consisting of n trials and results in x successes. If the probability of success on an individual trial is P, then the binomial probability is: b(x; n, P) = nCx * Px * (1 - P)n - x What is the Probability Coin Flip Nostradamus Predicts at least 3 of 4 Coin Tosses ?
  18. 18. 4! Coin Flip Nostradamus: Binomial Distribution (3! (4-3)! )(.53 ) (.54-3 ) (.125) (.5) 24 ( 6(1) ) = .25 Here is the Prob of Getting Exactly 3 of 4 correct
  19. 19. 4! Coin Flip Nostradamus: Binomial Distribution (3! (4-3)! )(.53 ) (.54-3 ) (.125) (.5) 24 ( 6(1) ) = .25 Here is the Prob of Getting Exactly 3 of 4 correct = .3125 We Want “At Least” Which Implies BOTH 3 and 4 .25 + .0625 Exactly 3 Exactly 4 at least 3 of 4 Coin Tosses
  20. 20. Namely, there is a 31.25% Probability that by Chance he/she would be able to predict at least 3 out of 4 If Our Would Be Coin Flip Nostradamus were able to get 3 out 4 Correct - we would not generally be prepared to give him/her credit just yet Coin Flip Nostradamus: Binomial Distribution
  21. 21. How Much Do We Need to Be Convinced that Our Friend is Actually Coin Flip Nostradamus? Now We Can Calculate Probability Associated of Prediction across some arbitrary number of trials Coin Flip Nostradamus: Binomial Distribution This is a Question of Type I and Type II Error
  22. 22. Type I v. Type II Error
  23. 23. Type I v. Type II Error
  24. 24. Type I v. Type II Error It is Depends Upon the Application Typical Convention is that a 5% Chance of Error is Acceptable for Purposes of Statistical Significance Social Science = 5% Medicine with Serious Side Effects might Require Greater Level of Significance 1% or even less
  25. 25. Back To Coin Flip Nostradamus Predicts 43 out of 75 Correct Okay let say Our Coin Flip Nostradamus agrees to run 75 coins flips in order to demonstrate his/her true powers Is this Sufficient to Label Our Friend the Coin Flip Nostradamus?
  26. 26. Binomial Probability Calculator http://stattrek.com/tables/binomial.aspx
  27. 27. Binomial Probability Calculator http://stattrek.com/tables/binomial.aspx Enter These Three Values + Hit Calculate
  28. 28. Binomial Probability Calculator http://stattrek.com/tables/binomial.aspx And These are the Results Our P value Here is 12.4%
  29. 29. Coin Flip Nostradamus In this Case, the P Value is Our P Value is the Probability of Observing this Data Given the Null (i.e. that our friend does not have psychic powers) Our Pvalue > 5% Statistical Significance Threshold “Fail to Reject” Our Null of No Psychic Powers (We Do not Say Accept -- see the induction problem)
  30. 30. One Tailed -or- Two Tailed Tests In the Coin Flip Nostradamus Example it would be amazing if our friend could actually fail to predict 75 consecutive events There is a Difference Between a Directional and a Non- Directional Hypothesis Note: These are Symmetric
  31. 31. One Tailed -or- Two Tailed Tests Stricter Crime Law and the Crime Rate We are Often Interested in a Non- Directional Hypothesis We are Interested in Whether there is Deterrence and if there were to be higher crime rates New Drug and Health We Want to Both if It Makes the Patient Better and if the Patient’s condition get worse
  32. 32. One Tailed -or- Two Tailed Tests Two Tailed Test One Tailed Test (negative direction) One Tailed Test (Positive direction)
  33. 33. https://onlinecourses.science.psu.edu/stat500/book/export/html/43 An Example of a Hypothesis Test Note: π is Prob α is the Significance Level
  34. 34. https://onlinecourses.science.psu.edu/stat500/book/export/html/43 An Example of a Hypothesis Test Want to Make Sure Sample is Large Enough Note: π is Prob α is the Significance Level
  35. 35. https://onlinecourses.science.psu.edu/stat500/book/export/html/43 An Example of a Hypothesis Test Want to Make Sure Sample is Large Enough If you Do Equal vs. Does Not Equal -- Two Tail Note: π is Prob α is the Significance Level
  36. 36. An Example of a Hypothesis Test https://onlinecourses.science.psu.edu/stat500/book/export/html/43 z = (p - P) / σ where p is our sample prov P is theorized population prob σ is our Standard Deviation
  37. 37. An Example of a Hypothesis Test https://onlinecourses.science.psu.edu/stat500/book/export/html/43
  38. 38. I roll a single die 1,000 times and obtain a "6" on 204 rolls. Is there significant evidence to suggest that the die is not fair? Another Example Question
  39. 39. Another Example Question
  40. 40. Daniel Martin Katz @ computational computationallegalstudies.com lexpredict.com danielmartinkatz.com illinois tech - chicago kent college of law@

×