2. Common Mistakes in Social Research
1. Contamination
2. Fallacies of presumption
– Hasty Generalization
– False Dichotomy
– Spurious association
– ‘Post hoc’ fallacy
3. Fallacies of the wrong level
– Ecological Fallacy (group to individual)
– Reductionist Fallacy /Fallacy of Composition (individual to
group)
4. ‘Ad Hoc’ Fallacies
3. I. Problem of Contamination
• Suppose that there is so much heat given off in the
first test tube, Y₁ that it spreads and heats up Y₂ . This
is contamination!
• The Error of Contamination occurs when the social
researcher acts as if the influence of an independent
variable is restricted solely to experimental group
when in fact it is also influencing the ‘control group’.
Influencing the control Y₂
Y₁
4. I. Problem of Contamination
• One cannot assume that a change in an
independent variable (X) affects the
dependent variable (Y) only in those settings
where the independent (X) variable is present
or has changed.
• Why? Because people observe what happens
elsewhere. The mere existence of some X in
some setting, may affect Y in other settings
where X isn’t present, or has changed.
5. I. Problem of Contamination
Example: Sweden v. Norway
• The effect of Norway’s entrance
into World War II (X) on fertility
rates in Norway (Y₁), using Sweden
(Y₂) as a control.
• An invalid inference might be: “If
Norway had not been invaded in
1940, its fertility rates would have
been like Sweden’s at that time”
6. I. Problem of Contamination
Example: Sweden v. Norway
• One problem (among many) is that
Sweden is not a good ‘control’, even if it is
exactly like Norway in all other
conceivable characteristics, and even if it
was an exact replica of Norway.
• Because Sweden was also affected by the
Nazi invasion…
7. II. Fallacies of Presumption
1. Hasty generalization: making a general
conclusion based on too little information
– My former husband was a jerk…from that I learned
that all men are jerks.
2. False Dichotomy (also called “False Bifurcation”,
“Black and white fallacy;” “either/or fallacy”
“False dilemma.” ): involves turning a complex
issue into one that has only two choices that are
opposite of one another
– ‘You are either with or against us!’
8. II. Fallacies of Presumption
3. Fallacy of false cause (spurious association)
– It says (wrongly) that if two things are associated,
then one of them must be the cause of the other.
If A and B are associated, then A must cause B.
– Example: More and more young people are
attending high schools and colleges today than
ever before. Yet there is more and more juvenile
delinquency among the young than every before.
This makes it clear that these young people are
being corrupted by their education.
9. II. Fallacies of Presumption
3. Fallacy of false cause (spurious association)
• ‘Post hoc’ fallacy: a more specific form of spurious
association, which asserts that, if A occurs before B,
then A is necessarily the cause of B.
– Derived from the Latin phrase,“Post hoc, ergo propter
hoc” (Latin: After this, therefore because of this).
– Example 1: 98% of Heroin users started off with
marijuana. Therefore, marijuana smoking causes people
to go on to the hard stuff.
– Even more drank alcohol, and 100% drank water! Only
about 1% of marijuana users end up using heroin.
10. II. Fallacies of Presumption
3. Fallacy of false cause (spurious association)
• ‘Post hoc’ fallacy:
• Example 2: Dr. Manfred Sakel discovered in 1927 that
schizophrenia can be treated by administering overdoses
of insulin, which produced convulsive shocks. Hundreds of
psychiatrists drew a faulty conclusion and began to treat
schizophrenia and other mental disorders by giving
patients electric shocks without insulin. So, they skipped
the insulin but went to shocks. At a psychiatric meeting
some years later, Dr. Sakel sadly came forward to explain
that electric shocks are actually harmful, while insulin
treatment restores the patient’s hormonal balance. The
doctors had confused the side effect with a cause.
11. III. Fallacies of the Wrong Level
• Ecological fallacy: studying something with
the group as the unit of the analysis and
making inferences about the individual
– Group Individual
• Reductionistic fallacy: studying something
with the individual as the unit of analysis and
making inferences about the group
– Individual Group
12. III. Fallacies of the Wrong Level
Ecological fallacy: (inferring lower from higher
levels, or parts from wholes)
• Example 1: In the United States presidential
elections of 2000, 2004, and 2008:
i. Wealthier states tended to vote Democratic
ii. Poorer states tended to vote Republican.
iii. Yet wealthier voters tended to vote Republican
and poorer voters tended to vote Democratic.
• The error would be to assume that, because
wealthier states voted Democratic, wealthier
voters also tended to vote Democratic.
13. III. Fallacies of the Wrong Level
Ecological fallacy: (inferring lower from higher
levels, or parts from wholes)
• Example 2: In American cities, there is a strong
relationship between illiteracy rate and
proportion of people who are foreign born. Does
this association hold for individuals? No, it could
be that all the foreign born are highly literate,
they just gravitate to urban areas where there are
also lots of native born people who are illiterate.
14. III. Fallacies of the Wrong Level
Ecological fallacy: (inferring
lower from higher levels, or
parts from wholes)
Example 3:
• Suppose you flip 10 unbiased
coins 5 times.
• A count of all of the coin tosses
will be pretty close to 25 heads
and 25 tails or 50-50%.
15. III. Fallacies of the Wrong Level
Ecological fallacy: (inferring lower from
higher levels, or parts from wholes)
Example 3:
• Some coins, however, will have more
heads than tails, others will have more
tails than heads, for entirely random
reasons.
• We cannot infer from the overall
distribution of heads and tails (50-
50%), the specific distribution of heads
and tails for each coin!
16. III. Fallacies of the Wrong Level
Reductionist Fallacy (aka Fallacy of composition):
(inferring higher levels from lower levels, or the
whole from the parts):
• Example 1: ‘Paradox of Thrift.’ Saving is good for
an individual, but not necessarily for the
economy as a whole, because lack of spending in
the aggregate can cause a recession.
• The error would be to assume that what
individual interest necessarily coincides with
collective or aggregate welfare.
17. III. Fallacies of the Wrong Level
Reductionist Fallacy (aka Fallacy of composition):
(inferring higher levels from lower levels, or the
whole from the parts):
• Example 2: ‘Dream Team’: Consider the study of
basketball. Suppose you gather together the best
players in the world. Does this mean that your
team will naturally be the best? Not necessarily.
(All these great players might have such great
egos that they can’t manage to play together; a
team of mediocre players might click so well that
they are unbeatable as a team).
18. IV. Non Sequitur
• Non sequitur fallacy (non-SEK-wa-tuur): the
term is Latin for “it does not follow.”
• In logic, the term is used to indicate a
conclusion that can not be justified by the
premises or evidence offered in an argument.
In other words, the non sequitur fallacy occurs
when an the conclusion does not follow from
the premises.
19. IV. Non Sequitur
Argument A:
(1) Most poor people don’t commit crimes
(2) Some rich people commit crimes
• Therefore, there is no relation between poverty
and crime!
Argument B:
1. Most people with bullet wounds don’t die.
2. Some people without bullet wounds do die.
• Therefore, bullet wounds are not a direct cause
of death. ?????
Editor's Notes
One can observe that a certain difference exists and that it is caused by a certain condition, but one cannot infer from this difference what would have occurred had the test condition been absent. (Lieberson 1985: 55).
See (Lieberson 1985: 56)
See
Sometimes what is most important is the overall distribution or pattern, and not the components that make up the pattern. Fundamental or basic driving forces tend to generate overall patterns, but leave undetermined its specific manifestation. For example, we know that in an educational setting, grades are a selection mechanism, and courses are geared towards generating certain overall outcomes, such as a grade distribution. Once we know that not everyone will make an A, by design, then we are less likely to attribute the overall distribution exclusively to the attributes (successes or failures) of the individual students.