Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Quantitative Methods for Lawyers - Class #5 - Research Design Part V - Professor Daniel Martin Katz


Published on

Quantitative Methods for Lawyers - Class #5 - Research Design Part V - Professor Daniel Martin Katz

Published in: Law, Technology, Business
  • Be the first to comment

  • Be the first to like this

Quantitative Methods for Lawyers - Class #5 - Research Design Part V - Professor Daniel Martin Katz

  1. 1. Quantitative Methods for Lawyers Research Design - Part V Class #5 @ computational professor daniel martin katz
  2. 2. The Origins of Modern Research Design / Statistical Inference
  3. 3.
  4. 4. Correlation vs. Inference
  5. 5.
  6. 6.
  7. 7. 
 A higher education (independent variable) typically leads to a higher income (dependent variable). 
 There is a correlation between education and income. 
 This allows people to say they make a great income because of their great education. Example:
  8. 8. 
 Occupation is a potential intervening variable.
 Between education and income is occupation. The level of a person’s education affects the chances for a good occupation, and that occupation then affects the income. 
 Education does not directly cause income an indirect cause, but not a direct cause.
  9. 9. 
 Occupation is a potential intervening variable.
 Between education and income is occupation. The level of a person’s education affects the chances for a good occupation, and that occupation then affects the income. 
 Education does not directly cause income an indirect cause, but not a direct cause. Indep. variable > intervening variable > dep. variable [education > occupation > income
  10. 10. Another Example: Ice Cream Sales and Crime Rate ?
  11. 11. 
 Extraneous variables are defined as any variable other than the independent variable that could cause the change in the dependent variable. 
 Extraneous (as Independent var) > dependent variable [ Ice Cream Sales > deaths ]
  12. 12. Visual of the Example: IceCream Consumption Crime Heat
  13. 13. 
 One group uses the researcher’s new learning strategy. The other group uses a strategy of their choice. Finally, all students are tested over the materials. The psychologist testifies as to the benefits of this new learning strategy. 
 How does the attorney challenge this study?
 An educational psychologist develops a new learning strategy. First, the experimenter randomly assigns students to two groups. Second, both groups study text materials for thirty minutes on a biology topic.
 Concept: The ecological fallacy arises when group data is used to draw conclusions about individuals. The fallacy arises when an individual assumes something is true of one or more of the parts because it is true of the whole.
  16. 16. For example, the average income of residents in a particular region is $40,000. Closer examination of the area shows the neighborhood is actually composed of two housing estates. One is a lower socio-economic group of residents, and the other is a higher socio-economic group. The residents in the poorer part of town earn on average $13,000 while the more affluent citizens average $61,000. the average in this example is constructed from two disparate groups. In the end, at the individual level, it actually is unlikely that any one person in the group earns $40,000.
  17. 17. The Schelling Social Segregation Model
  18. 18. Schelling Social Segregation Model
  19. 19. Schelling Social Segregation Model
  20. 20. Schelling Social Segregation Model
  21. 21. Schelling Social Segregation Model
  22. 22. Schelling Social Segregation Model
  23. 23. Schelling Social Segregation Model
  24. 24. Schelling Social Segregation Model versus note: stopped model before % unhappy = 0
  25. 25. Schelling Social Segregation Model Test these Parameters and report Results
  26. 26. Schelling Social Segregation Model
  27. 27. Schelling Social Segregation Model
  28. 28. Schelling Social Segregation Model
  29. 29. Schelling Social Segregation Model versus
  30. 30. What is the micro to macro disconnect? relationship to the ecological inference fallacy?
  31. 31. Question: Assume that statistics show that the largest number of robberies occurs in the downtown district from the studied city. Assume also that the census shows that the average incomes of persons living in that downtown district are substantially higher than any other area in the city. Can we conclude that in the downtown region, the high income residents are more likely to be robbed?
  32. 32. It is tempting to conclude that high income residents are more likely to be robbed in that location, but this data does not establish who was robbed. The group data only establishes that rich people live in a high crime area. The robbers might select rich victims or they might select lower income people commuting into the downtown to work.
  34. 34. Reliability is the degree of consistency. For example, when a group of people weigh themselves twice on the same scale, if the scale readings are consistent between the first and second weighing, there is total reliability. 
 Suppose a group of people weigh themselves twice, once on the first scale and then on a second scale. Suppose the recorded rates vary between the two scales. In such an example, there is variance, and as such, a measurable lesser degree of reliability. Again, reliability is a measurement of consistency.
  35. 35. Concept: Validity is a question of truthfulness. 
 Does the test actually measure what it purports to measure? Using the prior example, does the scale actually weigh the true weight of the individual?
  36. 36. If a single bathroom scale cannot even give a consistent weight, it is certain not to be accurate. 
 In such a case, that single bathroom scale lacks both reliability and validity because that single scale is inconsistent, and that single scale does not read accurately.
  37. 37. Question: Suppose a group of people test on two scales and those the two scales are consistently five pounds apart. Is there reliability? Can there be validity?
  38. 38. Two scales consistently read five pound apart. Looking at this result from the standpoint of each scale, the two scales are reliable. 
 In other words, one scale is always five pound heavier and therefore is always consistent within it. The same reliability exists for the other scale which is always five pound lighter. 
 Still, both scales cannot be truthful, but it is possible in this example, that one scale is truthful. Possibly, one scale has validity and reliability, but both scales cannot have validity.
  39. 39. Internal validity asks how valid is the research? 
 The answer to this question lies within the research project. Examples of threats to internal validity are selection bias, incorrect recording of data, and participants dropping out of the study. Examples of control against these threats are random selection and a narrow study.
  40. 40. 
 External Validity addresses how much the researcher can generalize the outcome to the greater population. In more plain words, just how broadly can the outcome of this study be read? 
 All researchers want to generalize the outcome as broadly is possible.
  41. 41. 
 Challengers to this early smoking study could legitimately question, are moderate smoking levels safe? Without breaking the data down by smoking qualities, the answer is unknown. 
 By restricting this early study to two general variables of cigarette smokers and lung diseases, there was no generalization capacity to show outcomes to societal populations with multiple variables. This is an example of a threat to the internal validity because of variable selection error. If the internal validity is not solid, there cannot be external validity.
  42. 42. Describe some of the RELIABILITY AND VALIDITY Issues associated with the LSAT? The Bar Exam?
  43. 43. Surveys
  44. 44. There are four basic types of surveys: 1) mail, 2) telephone, 3) online, and 4) in person.  Mail surveys are paper and pencil documents that are mailed to respondents.  They are self-administered by the recipient. Based upon the fact that the questionnaire is filled out by the recipient, what is the strength and weakness of that point?
  45. 45. Answer: The researcher has little control over the answer. The ability to clarify the answer is less since the respondent has no person present to ask clarifying questions. The absence of a researcher representative can result is erroneous responses. On the other hand, the recipient might feel a sense of anonymity which allows for more correct responses to sensitive questions.
  46. 46. Surveys by telephone might be conducted by trained interviewers.  Data collected through telephone surveys usually has minimal missing or erroneous data primarily because it offers the opportunity for personal assistance.  Those interviewers typically follow a regularized protocol or an actual script. New automated random dialing systems increase the “randomness of the sample.” but there are still additional Issues ...
  47. 47. In today’s world, what is the current downfall for researchers trying to contact respondents through telephones?
  48. 48. In 2011, land line telephones were unavailable to about a third of all households. This shows the great rise of cell phones as the primary phone system. Further, scientific researchers using automated phone surveys are barred by “do not call” restrictions. Finally, many phone respondents have the capacity to advance read the caller ID. They simply do not answer. On a national basis, the use of the telephone survey may no longer be the predominate method for surveys
  49. 49. Online surveys are experiencing the largest growth. However, it still has some limitations. Why?
  50. 50. Surveys can also be administered by computer and the Internet.  All provide the potential to conduct complicated research because “help menus” can assist respondents through the survey.  The questionnaires also can include visual aids or images as part of those surveys.  And perhaps most importantly, they are the least expensive format. To achieve validity, however, they are best used with respondents who are pre-recruited. Otherwise, predicting the responding sample can be difficult. By definition, online surveys are only available to those with access to computers. Also, the surveys can be filtered by programs barring unwanted span. The big issue involving online surveys is “what determines your sample?”
  51. 51. Drafting questions for a scientific questionnaire is a methodological process. There are steps to be considered prior to drafting a single question. Where do you think the researcher should start this process?
  52. 52. Questions in a survey should have an internal validity check. Suppose the hypothesis is that the educational experience in a specific, demographically changing community is characterized by a high level of educational instability. The researcher decides this community’s educational instability is a result of rapid population growth which corresponds to a rapid growth in the school population. The research hypothesizes that this rapid growth developed instability in the education plan as educators struggled to deal with the rapidly growing school population. As stated, one of the researcher’s pre-determined specific characteristics is a rapid growth.
  53. 53. Give three objective questions for the questionnaire which will allow the researcher to validate the portion of the hypothesis which claims a rapid growth in the school population.
  54. 54. Three objective indicator questions of this rapid growth hypothesis element are: (1) Total student enrollments of the respondent’s school for each of the last 3 years. (2) The number of teachers in the school for each of the last 3 years. (3) Dollar figures for construction of the school for each of the last 3 years.
  55. 55. By asking those three objective questions in the questionnaire, the researcher can help validate the portion of the hypothesis which claims a rapid growth of the school population. If all three answers show substantial growth in each of the last three years, this element will be validity by those three different approaches: student enrollments, teacher numbers and construction expenses.
  56. 56. Assume the above objective indicator questions validated the school’s rapid population growth. The researcher now wants objective questions to validate another element of the hypothesis, that is, this rapid growth caused some educational instability. Give objective questions to act as an internal validity check for that educational instability element of the hypothesis?
  57. 57. Objective indicator questions of this educational instability element are:
 (1) What is the average number of years of teaching experience for employed teachers for each of the last three years? (2) What is the average classroom size for each of the last three years? (3) How many teachers had to be replaced after choosing to leave the school during the last three years?
  58. 58. The researcher decides the type of questions. They can be closed, open or contingent questions.
  59. 59. Closed Question How often do your parents question whether you have homework? (1) Never (2) Less than once per week (3) Once per week (4) Two or more times per week
  60. 60. Open Question What do you like most about school? Contingent Question Are you on the honor roll? (1) Yes … [If yes, answer the following question] (2) No … [If no, skip to question # 9]
  61. 61. What are some of the advantages / disadvantages of each question type?
  62. 62. As a general rule, questions in the questionnaire should be short and concrete. Do not combine concepts. For example, do not ask: “Do you think the school should teach geography and art?” Respondents do not know how to respond if they think yes to art and no to geography.
  63. 63. Be careful about historical questions. “How many times in the last 5 years did you get a flu shot?” Many historical questions are important, but faulty memories can make the answers into just guesses.
  64. 64. Leading questions can bias the data. “Do you oppose a longer school day?” This suggests the researcher’s opinion. The question should be, “Do you favor or oppose a longer school day?” Please note, there is a vague term. What is a longer school day? The question must be premised with a neutral definition.
  65. 65. Using Members of your target population PRETEST, PRETEST, PRETEST Your Questions !
  66. 66. Daniel Martin Katz @ computational illinois tech - chicago kent college of law@