buc

1. 1
Chapter 4:
General Issues in
Research Design
© 2018 Cengage Learning. All Rights Reserved.

2. 2
© 2018 Cengage Learning. All Rights Reserved.
Learning Objectives
• Recognize how explanatory scientific research centers
on the notion of cause and effect, and why this is a
probabilistic model of causation
• Describe the three basic requirements for establishing
a causal relationship in science, together with what is
a necessary cause and a sufficient cause
• Understand the role of validity and threats to validity of
causal inference
• Summarize the four classes of validity threats, and
how they correspond to questions about cause and
effect

3. 3
© 2018 Cengage Learning. All Rights Reserved.
Learning Objectives, cont.
• Discuss how a scientific realist approach bridges
idiographic and nomothetic approaches to causation
• Describe different units of analysis in criminal justice
research
• Explain how the ecological fallacy relates to units of
analysis
• Understand the time dimension, together with the
differences between cross-sectional and longitudinal
research
• Describe how retrospective studies may approximate
longitudinal studies

4. 4
© 2018 Cengage Learning. All Rights Reserved.
Introduction
• Causation, units, and time are key
elements in planning a research study
• As social scientists, we seek to explain the
causes of some phenomenon (e.g., crime)
• Who or what we are studying is an
important part of research
• Researchers also must consider the time
order of events

5. 5
© 2018 Cengage Learning. All Rights Reserved.
Causation in the Social Sciences
• Causation is the focus of explanatory research
• Cause in social science is inherently
probabilistic
• Certain factors make crime/delinquency more or less likely within
groups of people
• Two models of explanation
• Ideographic: Lists the many, perhaps unique considerations behind an
action
• Nomothetic: Lists the most important (and fewest) considerations/variables
that best explain general patterns of cause and effect

6. 6
© 2018 Cengage Learning. All Rights Reserved.
Criteria for Causality
• Posited by Shadish, Cook, & Campbell (2002)
• Empirical relationship between variables
• Temporal order (cause precedes effect)
• No alternative explanations—no spurious other variable(s)
affecting the initial relationship
• Any relationship that satisfies all these criteria is
causal

7. 7
© 2018 Cengage Learning. All Rights Reserved.
Necessary and Sufficient Causes
• Within the probabilistic model, two types:
• Necessary cause: Represents a condition that must be present
for the effect to occur (e.g., being charged is necessary cause to
be convicted)
• Sufficient cause: Represents a condition that, if it is present, will
pretty much guarantee that the effect will occur (e.g., pleading
guilty is sufficient cause to being convicted)

8. 8
© 2018 Cengage Learning. All Rights Reserved.
Validity and Causal Inference
• Scientists assess the truth of statements about
cause by considering threats to validity
• When we make a cause-and-effect statement, we
are concerned with its validity—whether it is true
and valid
• Certain threats to the validity of our inference exist
• These are reasons why we might be incorrect in
stating that some cause produces some effect

9. 9
© 2018 Cengage Learning. All Rights Reserved.
Discussion Question 1
What are the greatest threats to validity in
social science?

10. 10
© 2018 Cengage Learning. All Rights Reserved.
Statistical Conclusion Validity
• Refers to our ability to determine whether
a change in the suspected cause is
statistically associated with a change in
the suspected effect
• Are two variables related to each other?
• Researchers cannot have much
confidence in statements about cause if
their findings are based on a small number
of cases

11. 11
© 2018 Cengage Learning. All Rights Reserved.
Internal Validity
• An observed association between two variables
has internal validity if the relationship is, in fact,
causal and not due to the effects of one or more
other variables
• Generally due to nonrandom or systemic error
• The threat to IV results when the relationship
between two variables arises from the effect of
some third variable
• Example: drug users sentenced to probation over prison
recidivate less

12. 12
© 2018 Cengage Learning. All Rights Reserved.
Discussion Question 2
How can you best set up an experiment with
strong internal validity?

13. 13
© 2018 Cengage Learning. All Rights Reserved.
External Validity
• Concerned with whether research findings in
one study can be replicated in another study,
often under different conditions
• Do the findings apply equally in different settings
(locales, cities, populations)?
• Kansas City evaluation found sharp reductions
in gun-related crimes in hot spots that had been
targeted for focused police patrols
• Indianapolis and Pittsburgh launched similar projects

14. 14
© 2018 Cengage Learning. All Rights Reserved.
Construct Validity
• Concerned with how well an observed
relationship between variables represents the
causal process
• Refers to generalizing from what we observe
and measure to the real-world things in which
we are interested
• e.g., close supervision of officers -> more tickets?
• e.g., Kansas City Preventive Patrol Experiment, “police visibility”

15. 15
© 2018 Cengage Learning. All Rights Reserved.
Validity & Causal Inference Summarize
• The four types of validity threats can be
grouped into these two categories
• Bias: Internal Validity and Statistical
Conclusion Validity threats are related to
systematic and nonsystematic bias
• Generalizability: Construct Validity and
External Validity are concerned with
generalization to real-world behaviors and
conditions

16. 16
© 2018 Cengage Learning. All Rights Reserved.
Discussion Question 3
What if someone offered you a survey taken
by South Africans to help you with your
survey project for North Americans? Would
you have any reservations as a social
scientist?

17. 17
© 2018 Cengage Learning. All Rights Reserved.
Does Drug Use Cause Crime?
• Temporal order: which comes first?
• A statistical relationship exists, but
underlying causes affect both drug use and
crime (Internal Validity threat)
• What constitutes drug use? Crime?
(Construct Validity threat)
• How will policy affect drug use and crime?
• A crackdown on all drugs among all populations will do
little to reduce serious crime

18. 18
© 2018 Cengage Learning. All Rights Reserved.
Introducing Scientific Realism
• Bridges idiographic and nomothetic
approaches to explanation by seeking to
understand how causal mechanisms operate
in specific contexts
• Studies how such influences are involved in cause-and-
effect relationships
• Exhibits both ideographic & nomothetic approaches to
explanation
• "Can the design of streets and intersections be modified to
make it more difficult for street drug markets to operate?"

19. 19
© 2018 Cengage Learning. All Rights Reserved.
Units of Analysis
• What or who is studied
• Individuals: Police, victims, defendants, inmates, gang
members, burglars, etc.
• Groups: Multiple persons with same characteristics (gangs,
cities, counties, etc.)
• Organizations: Formal groups with established leaders and
rules (prisons, police departments, courtrooms, drug
treatment facilities, etc.)
• Social artifacts: Products of social beings and their
behavior (stories in newspapers, posts on the Internet,
photographs of crime scenes, incident reports,
police/citizen interactions)

20. 20
© 2018 Cengage Learning. All Rights Reserved.
Issues of Logic
• Ecological fallacy: Danger of making assertions
about individuals based on the examination of
groups or aggregations
• Poor areas = more crime, therefore poor people commit more
crime
• Individual fallacy: Using anecdotal evidence to
make an argument
• O.J. Simpson court resources
• Reductionism: Failing to see the myriad of
possible factors causing the situation being
studied

21. 21
© 2018 Cengage Learning. All Rights Reserved.
The Time Dimension
• Time sequence is critical in determining
causation
• Time is also involved in the generalizability
of research findings
• Observations can either be made more or
less at one point, or stretched over a
longer period
• Observations made at more than one time point can
look forward or backward

22. 22
© 2018 Cengage Learning. All Rights Reserved.
Cross-Sectional Studies
• Observing a single point in time (cross-section)
• Simple and least costly way to conduct research
• Typically descriptive or exploratory in nature
• A single wave of the National Crime
Victimization Survey (NCVS) is a descriptive
cross-sectional study that estimates how many
people have been victims of crime in a given
time

23. 23
© 2018 Cengage Learning. All Rights Reserved.
Longitudinal Studies
• Permit observations over time
• Trend: Those that study changes within some general
population over time (UCR)
• Cohort: Examine more specific populations as they
change over time (Wolfgang study)
• Panel: Similar to trend or cohort, but the same set of
people is interviewed on two or more occasions
(NCVS, panel attrition)

24. 24
© 2018 Cengage Learning. All Rights Reserved.
Approximating Longitudinal Studies
• Gun ownership and violence study by Swiss
researcher Martin Killias (1993)
• Compared rates of gun ownership as reported in an international
crime survey to rates of homicide and suicide committed with
guns
• May be possible to draw approximate
conclusions about processes that take place
over time, even when only CS data is available
• When time order of variables is clear, logical
inferences can be made about processes taking
place over time

25. 25
© 2018 Cengage Learning. All Rights Reserved.
Retrospective Research
• Asks people to recall their past for the
purpose of approximating observations over
time
• People have faulty memories; people lie
• Analysis of past records also suffer from
problems—records may be unavailable,
incomplete, or inaccurate
• Prospective research: longitudinal study that
follows subjects forward in time (Widom, child
abuse/drug use)

26. 26
© 2018 Cengage Learning. All Rights Reserved.
Time Dimension Summarized
• Cross-sectional study = snapshot: an
image at one point in time
• Trend study = slide show: a series of
snapshots in sequence over time, allows
us to tell how some indicator varies over
time
• Panel study = motion picture: gives
information about individual observations
over time