Quantitative Methods for Lawyers - Class #3 - Research Design Part III - Professor Daniel Martin Katz
Research Design - Part III
professor daniel martin katz danielmartinkatz.com
A sample is representative when it is an accurate
proportional representation of the population under
Representativeness is a
contingent concept that
must considered relative
to the overall population
Sample size is dependent
upon variation and risk.
Because we need to
on all relevant variables
For example, a study which only
examines a gender distinction will
probably be successful with a smaller
sample size than a study with a large
number of variables, such as age,
income, race, education, etc.
Risk involves amount
toleration for error.
Normally, a social science
researcher tolerates up to a
5% chance of random error.
If the study involves medicine where
life and health is at issue, there is
little toleration for error. The study
would then require a large sample.
All else equal, greater accuracy is
obtained with larger sample sizes
and a lower levels of accuracy are
obtained with a smaller sample sizes.
Question: The researcher randomly selects 100 murder
trials prosecuted in this state over the last six years.
The researcher seeks to evaluate whether court
appointed legal representation disproportionately
enhances the likelihood of a death penalty outcome.
The analysis suggests lower death penalty outcomes
when attorneys are privately hired. A prosecutor seeks
to challenge this study claiming the sample as invalid.
Name some potential arguments can the prosecutor
assert against the sampling approach?
The researcher studied 100 murder
trials over a 6 year period. It is
important for the trial lawyer to establish
if the sample was sufﬁciently large and
sufﬁciently reﬂective of the greater
population. The attorney can raise many
questions about the sample.
What percentages of the sample involved court appointed attorneys and
privately funded attorneys?
How do those percentages compare to the full population of murder trials
within that state?
Did the sample of 100 trials represent a particularly small or particularly
large size when compared to the full population of murder trials from that
For instance, is there any category in the data which measures a
disproportional death penalty impact solely because the murder was
particularly gruesome and hideous?
Here are just a few:
Did the sample consider the strength and experience of the attorneys
regardless if they were court appointed or privately funded?
Did the sample consider whether some trials were high proﬁle?
Did the sample consider race and income of the defendants?
And a few more:
Concept: In most instances, it is
impractical and overly expensive to
study every member of the population.
The researcher must seek to obtain a
representative sample of that greater
population. It is a two part process.
Example: Should a researcher
elect to study death penalty
data, the researcher must deﬁne
the limits of that population.
Is the population limited to those
actually sentenced to death or is the
population broader to include those
who were eligible under the law to be
sentenced to death?
Or, is the researcher using the
population of those who could have
been sentenced to death but were not?
Here are the Most Common Approaches
1. Simple random
3. Stratiﬁed random
4. Matched Pairs
Each individual (or object) is chosen randomly and entirely by
chance, such that each individual has the same probability of being
chosen at any stage during the sampling process
Replacement vs. Non-Replacement ?
Quick Word on
We have patterns in
our behavior even
when we think we are
Try this Applet
A stratiﬁed sampling technique is used to
insure that targeted members of the selected
population are included in the sample.
The researcher divides the entire
target population into different
subgroups called strata.
The researcher then randomly selects
from all of the different strata.
For example, to obtain a stratiﬁed sample of
university students, the researcher might ﬁrst
organize the population by college class and
then randomly select from each strata (i.e. an
appropriate number of freshmen, sophomores,
juniors, and seniors).
Concept: Matched pairs is an example of a “related design.”
It is used commonly with experiments or to mimic the properties
of an experiment.
Participants are matched on variables considered very relevant
For example, pairs might be matched on scores from a health
test or personality tests.
Matched pairs is a sampling technique commonly used with
Example: The prosecution’s expert at a DUI sentencing. The
expert’s testimony concerns a research experiment on that
In that experiment, the effect of drinking and driving is
demonstrated by two groups of people driving around a
pre-selected course with speciﬁed amounts of alcohol in
The control group has no alcohol and the experimental
group drives the same course after consuming a speciﬁed
amount of alcohol.
As the lawyer objecting to this testimony, what serious
research problem will you assert about these two groups?
The participants from the two groups may signiﬁcantly vary
in personality, age, sex, cognitive ability, attention span,
etc. IF the researcher did not match those groups to
establish similarity between the groups on variables deemed
important to the experiment.
For example, If one group consists of elderly drivers while
the other group is younger drivers, is the experiment
measuring the manipulated alcohol variable or is the
experiment measuring the impact from age differences?