This document discusses reasons why research and researchers may not always provide truthful information. It notes that researchers can make mistakes, talk to the wrong samples, confuse correlation with causation, be biased in their methods and conclusions, and have conflicts of interest. The document also discusses why people in general may not always tell the truth, such as wanting to appear likable, boastful, avoid feeling stupid, or win something. It provides examples of misleading practices like leading survey questions, selective data analysis and reporting, and improper data visualization. The conclusion advocates handling data with care.

1. Why research (and researchers)
won’t always tell you the truth

2. Why researchers sometimes go wrong
•
They make mistakes
•
They talk to the wrong people (e.g. they can’t talk
to people without landlines on CATI surveys)
•
They confuse cause with effect (A and B regularly
happen together – therefore A causes B)
•
They are sure of the conclusion before the research
– & design the research (or fudge the data) to prove it
•
They are frightened of the wrong conclusions
•
They have reasons to lie (e.g. ties to funding
organisations)

3. Questionable research practices are common
John et al. (2011) Prevalence of Questionable Research Practices. Psychological Science 23(5) 524–532
3

4. Why don’t people always tell the truth?
•
They want to be liked
•
They are boasting (especially on social media!)
•
They don’t want to feel stupid
•
They hope to win something
•
They get bored
•
They can’t be bothered to think
•
Or perhaps they do tell the truth but they are not a
representative sample
•
Social media participants and survey respondents
are often “people with something to say”

5. How not to tell the truth in surveys
Sir Humphrey Appleby knew all about leading questions!
http://www.youtube.com/watch?v=G0ZZJXw4MTA

6. Selecting the data
•
What is the “best” set of data to fit my thesis
•
Are there “outliers” I can ignore – or include

7. Selecting the axes
7

8. Data visualisation – the wrong chart
Pie charts can easily cause
problems
And infographics can be very difficult to get right
(area of B on the left is 9x that of B on the right)

9. Selecting what you report
What is the average income in my street?
1 teacher
earning
One celeb earning
14 managers
£35,000
£4,464,000
making £50,000
20 other people only
Five professionals
making £10,000
making £150,000
mean = £150,000; median = £35,000; mode = £10,000

10. Conclusions
Handle data with care
Business is an art, not a science!
jeremy@mosoco.co.uk
10