http://asqbuffalo.org We should only believe things for which there is adequate evidence. But what evidence is adequate? We decide based on our experiences, beliefs, judgments, and reasoning. However, we often commit logical fallacies (errors in reasoning). When this happens either sound evidence is rejected or questionable evidence is accepted while leaving us feeling that we have made the right decision. Recognizing these errors can help us avoid making wrong decisions and promote data based decision making.

1. Skepticism
At Work
Recognizing
Logical
Fallacies
Steve Cena
2013

2. Basic Definitions
Cynicism: To doubt without evidence. To dismiss a claim out of
hand.
Skepticism: To doubt based on the evidence or the lack thereof.
Faith: To believe without evidence or in spite of contrary
evidence. “Faith is believing what you know ain’t so.”*
Belief: To accept based on the available evidence.
* Mark Twain

3. “I’m a skeptic not
because I do not want
to believe but I want to
know.”
Michael Shermer, The Believing Brain Prolog

4. Discussions of belief & skepticism are
often centered on paranormal or
pseudoscientific topics

5. Business Claims
• We need a fresh set of eyes to work on this
problem.
• We tried that before & it didn’t work.
• The test is 99% accurate.
• It’s just an outlier.
• I can’t explain why but whenever we do it this
way it works.

6. Which Claims Should We Believe?
WE SHOULD ONLY
BELIEVE THINGS FOR
WHICH THERE IS
ADEQUATE EVIDENCE.

7. What is adequate evidence?
DATA
How do we evaluate the data?
Judgments
Reasoning

8. Judgments = Opinions
Reasoning = Logic
Errors in Reasoning = Logical Fallacies
Claims are made on what seem to be good
arguments but these arguments are based
on incorrect logical principles. Thus, the
claims are not valid.

9. Ad Hominem Attack
Ad hominem is Latin for "to the person”
• Not invented here
• Outside expert error
• Guilty by group association

10. False Dichotomy
(The Excluded Middle)
Arbitrarily reducing the set of probabilities to only 2.
• “You are either for us or against us.”
• "You are either part of the solution or part of
the problem.”
• "It's either the design or the process."

11. Measurement Method Material
Response
or Problem
Machine Mother Nature Manpower

12. This
Response
or Problem
That

13. Anecdotes Do Not Equal Evidence
• Memory is fallible & malleable
– Memories are reconstructed not replayed
• Anecdotes rely on intuition & subjective
interpretation
• “… we are apt to want our version of the truth,
rather than the truth itself, to prevail.”*
• Anecdotex ≠ Data
– The plural of anecdote is not data.
• Show me the data
* Steven Pinker, How the Mind Works, pg. 289

14. Conjunction Fallacy
The tendency to think 2 conclusions are
more likely than either conclusion
alone.
• 80% of the time delivery is late
• 80% of the time the quantity is wrong
• What is more likely?
– The next delivery will be late
– The next delivery will be late and the wrong
quantity

15. Conjunction Fallacy
• 80% of the time delivery is late
– P(late) = .80
• 80% of the time the quantity is wrong
– P(quantity) = .80
• P(late) x P(quantity) = .80 x .80 = .64

16. Base Rate Fallacy
When considering the probability of
something happening failing to consider
the probability that it can happen.

17. Base Rate Fallacy
Test 100 acceptable parts
99 are correctly identified
1 incorrectly identified
Test 100 unacceptable parts
99 are correctly identified
1 incorrectly identified

18. Base Rate Fallacy
99% accurate!
99% of shipped product is acceptable!
99% of rejected product is unacceptable!
Only if half of the product made is unacceptable.

19. Base Rate Fallacy
Produce 10,000 units
95% are acceptable
10,000 x .95 = 9,500 acceptable units
10,000 – 9,500 = 500 unacceptable

20. Base Rate Fallacy
9,500 acceptable units tested at 99% accuracy
9,500 x .99 = 9,405 accepted
95 incorrectly rejected

21. Base Rate Fallacy
500 unacceptable units tested at 99% accuracy
500 x .99 = 495 rejected
5 incorrectly accepted

22. Base Rate Fallacy
What Shipped
9405 correctly accepted
+ 5 incorrectly accepted
9410 total accepted units
9405/9410 = .9995
Considering the BASE RATE
99.95% of shipped product is acceptable
NOT 99% as claimed.

23. Base Rate Fallacy
What is Rejected
495 correctly rejected
+ 95 incorrectly rejected
590 total rejected units
495/590 = .839
Considering the BASE RATE
83.9% of rejected product is unacceptable
NOT 99% as claimed.

24. Post Hoc Ergo Propter Hoc
Latin for "after this, therefore because of this“
• Assumption: if A precedes B then A must
cause B
• Coincidences happen
• Correlation ≠ Causation
– “New York City Has Most Millionaires In The
Country” *
• By What Mechanism?
* http://www.huffingtonpost.com/2010/08/04/new-york-city-has-most-
mi_n_670474.html#s122573&title=New_York_667200

25. Post Hoc Ergo Propter Hoc
XKCD

26. Regression to the mean
An extreme measurement will tend to be
followed by a measurement that is closer to
the mean.

27. Regression to the mean

28. Regression to the mean

29. Regression to the mean

30. Regression to the mean

31. Regression to the mean

32. Regression to the mean

33. Why do you believe?
• Is it because the belief is comfortable or
convenient?
• Does the data reaffirm what you already
"know" to be true?
• Are contrary data ignored or rationalized
away?
• Did you objectively analyze the data and come
to this conclusion?

34. What would change your mind?
• NOTHING!
• Falsifiable claim
– “The Scientific method consists of the use of procedures
designed to show not that our predictions and hypotheses
are right, but that they might be wrong”*
• Rabbit in the Precambrian**
What is your rabbit?
* Mistakes Were Made, Tarvis & Aronson
**J.B.S. Haldane

35. What would you do if you’re wrong?
• Minimize – It’s not that bad
• Mobilize – Moving target
• Optimize – (POB) Pervasive Optimistic Bias
• Rationalize – Justify
• Invoke any logical fallacy

36. Why do you believe?
What would change your mind?
What would you do if you’re wrong?

37. A Short Story
Rationalization
Straw Man Argument
Minimization
Mobilization
Argument From Authority
Argument From Ignorance
Optimize (POB)
Rationalization