Quantitative Methods for Lawyers - Class #3 - Research Design Part III - Professor Daniel Martin Katz


Published on

  • Be the first to comment

  • Be the first to like this

Quantitative Methods for Lawyers - Class #3 - Research Design Part III - Professor Daniel Martin Katz

  1. 1. Quantitative Methods for Lawyers Research Design - Part III Class #3 @ computational computationallegalstudies.com professor daniel martin katz danielmartinkatz.com lexpredict.com slideshare.net/DanielKatz
  2. 2. Sampling
  3. 3. Sampling Representative Sampling A sample is representative when it is an accurate proportional representation of the population under study Representativeness is a contingent concept that must considered relative to the overall population under study
  4. 4. Sample size is dependent upon variation and risk.
  5. 5. Variation

  6. 6. Variation refers to the amount of differences expected among the cases in the sample

  7. 7. Lets work through an example to better understand variation

  8. 8. Imagine We Had a Bag Full of Puzzle Pieces

  9. 9. Our Goal is to Make an Inference Regarding the Color(s) and/or Image(s) in the Overall Puzzle

  10. 10. As we draw them from the bag, each puzzle piece provides some information about color / images

  11. 11. How many pieces would you need to observe in order to feel confident regarding the color / images in the puzzle?

  12. 12. For example, imagine you drew these five puzzle pieces

  13. 13. Now make an inference regarding the “population” 

  14. 14. In other words, what does the puzzle look like? 

  15. 15. Like this?

  16. 16. Or Like this?

  17. 17. In general, the greater the variance in the population the larger sample you will need
  18. 18. Also, the fewer the number of significant variables, the smaller a sample can be.
  19. 19. Why?
  20. 20. Because we need to ensure representativeness on all relevant variables
  21. 21. For example, a study which only examines a gender distinction will probably be successful with a smaller sample size than a study with a large number of variables, such as age, income, race, education, etc.
  22. 22. Risk

  23. 23. Risk involves amount of acceptable toleration for error. 

  24. 24. Normally, a social science researcher tolerates up to a 5% chance of random error.
  25. 25. If the study involves medicine where life and health is at issue, there is little toleration for error. The study would then require a large sample.
  26. 26. All else equal, greater accuracy is obtained with larger sample sizes and a lower levels of accuracy are obtained with a smaller sample sizes.
  27. 27. Question: The researcher randomly selects 100 murder trials prosecuted in this state over the last six years. The researcher seeks to evaluate whether court appointed legal representation disproportionately enhances the likelihood of a death penalty outcome. The analysis suggests lower death penalty outcomes when attorneys are privately hired. A prosecutor seeks to challenge this study claiming the sample as invalid. 
 Name some potential arguments can the prosecutor assert against the sampling approach?
  28. 28. The researcher studied 100 murder trials over a 6 year period. It is important for the trial lawyer to establish if the sample was sufficiently large and sufficiently reflective of the greater population. The attorney can raise many questions about the sample.
  29. 29. 
 What percentages of the sample involved court appointed attorneys and privately funded attorneys? 
 How do those percentages compare to the full population of murder trials within that state? 
 Did the sample of 100 trials represent a particularly small or particularly large size when compared to the full population of murder trials from that state? 
 For instance, is there any category in the data which measures a disproportional death penalty impact solely because the murder was particularly gruesome and hideous? Here are just a few:
  30. 30. 
 Did the sample consider the strength and experience of the attorneys regardless if they were court appointed or privately funded? 
 Did the sample consider whether some trials were high profile? 
 Did the sample consider race and income of the defendants? 
 And a few more:
  31. 31. Sampling Techniques
  32. 32. Concept: In most instances, it is impractical and overly expensive to study every member of the population. The researcher must seek to obtain a representative sample of that greater population. It is a two part process.
  33. 33. First, Define the Relevant Population
  34. 34. 
 Example: Should a researcher elect to study death penalty data, the researcher must define the limits of that population.
  35. 35. 
 Is the population limited to those actually sentenced to death or is the population broader to include those who were eligible under the law to be sentenced to death? 
 Or, is the researcher using the population of those who could have been sentenced to death but were not?
  36. 36. Second, Select a Sampling Technique
  37. 37. Here are the Most Common Approaches 1. Simple random 2. Systematic 3. Stratified random 4. Matched Pairs
  38. 38. Each individual (or object) is chosen randomly and entirely by chance, such that each individual has the same probability of being chosen at any stage during the sampling process Simple Random See Also Replacement vs. Non-Replacement ?
  39. 39. 
 Quick Word on Randomness ...
 We have patterns in our behavior even when we think we are acting randomly 
 Try this Applet Rock Paper Scissors
 Randomness http://www.nytimes.com/interactive/science/rock-paper-scissors.html
  40. 40. Systematic Sampling
  41. 41. Systematic sampling normally draws a sample from every Nth case
  42. 42. The starting point or order is normally selected randomly, but the researcher pre-selects the interval of the remaining selections.
  43. 43. Watch out for some unknown component that generates bias.
  44. 44. For example, every select you select every 7th case and the data is organized by date
  45. 45. Stratified Sampling
  46. 46. 
 A stratified sampling technique is used to insure that targeted members of the selected population are included in the sample. 

  47. 47. 
 The researcher divides the entire target population into different subgroups called strata. 

  48. 48. 
 The researcher then randomly selects from all of the different strata.
  49. 49. 
 For example, to obtain a stratified sample of university students, the researcher might first organize the population by college class and then randomly select from each strata (i.e. an appropriate number of freshmen, sophomores, juniors, and seniors). 

  50. 50. Matched Pairs Design
  51. 51. Concept: Matched pairs is an example of a “related design.” 
 It is used commonly with experiments or to mimic the properties of an experiment. Participants are matched on variables considered very relevant For example, pairs might be matched on scores from a health test or personality tests. Matched pairs is a sampling technique commonly used with experimental designs.
  52. 52. Example: The prosecution’s expert at a DUI sentencing. The expert’s testimony concerns a research experiment on that topic. 
 In that experiment, the effect of drinking and driving is demonstrated by two groups of people driving around a pre-selected course with specified amounts of alcohol in their bodies. The control group has no alcohol and the experimental group drives the same course after consuming a specified amount of alcohol. 
 As the lawyer objecting to this testimony, what serious research problem will you assert about these two groups?
  53. 53. The participants from the two groups may significantly vary in personality, age, sex, cognitive ability, attention span, etc. IF the researcher did not match those groups to establish similarity between the groups on variables deemed important to the experiment.
 For example, If one group consists of elderly drivers while the other group is younger drivers, is the experiment measuring the manipulated alcohol variable or is the experiment measuring the impact from age differences? Matched Pairs
  54. 54. The Randomized Control Trial
  55. 55. Imagine a Study Involving Thirty Patients
  56. 56. We Would Like to Randomly Assign These Individuals to Two Groups Control Treatment
  57. 57. Then We Will Give the Treatment to the Treatment Group Treatment
  58. 58. Give Either Placebo or Baseline Treatment to the Control Group Control
  59. 59. Then, Compare Results that the Follow Control Treatment
  60. 60. Are These Differences Statistically Significant? Control Treatment
  61. 61. Randomized Control Trial Control Group Treatment Group Follow Up Evaluation Follow Up Evaluation
  62. 62. RCT’s Are Often Considered the Gold Standard in Science
  63. 63. Because if properly executed there is a fairly clean relationship between cause and effect
  64. 64. Daniel Martin Katz @ computational computationallegalstudies.com lexpredict.com danielmartinkatz.com illinois tech - chicago kent college of law@