3. Video Clip
(2:24 – 6:36)
• Scien)fic Studies: Last Week Tonight with John Oliver (HBO)
2016-10-17 | UC Berkeley
Alasdair Cohen | Lecture for Publich Health 250B
3
4. Growing Focus in the
Research Community
• “Research findings more likely to be false if…
• Study has a small sample size and may be underpowered
• The effect size is small
• A large number of effects are tested
• Studies with flexible designs, defini)ons, outcomes, analyses
• Studies for which there is great financial or other interest in the outcome”
2016-10-17 | UC Berkeley
Alasdair Cohen | Lecture for Publich Health 250B
4
Source: Jade Benjamin-Chung, 2016-05-04 presenta)on
to icddr.b Environmental Interven)ons Unit
5. Growing Focus in
Mainstream Media
2016-10-17 | UC Berkeley
5
Alasdair Cohen | Lecture for Publich Health 250B
9. P-hacking & a Brief
History of p-values
2016-10-17 | UC Berkeley
Alasdair Cohen | Lecture for Publich Health 250B
9
10. P-hacking:
How to do it?
• “Stop collec)ng data once p<.05
• Analyze many measures, but report only those with p<.05
• Collect and analyze many condi)ons, but only report those with p<.05
• Use covariates to get p<.05
• Exclude par)cipants to get p<.05
• Transform the data to get p<.05”
• …other ”methods”?
• Selec)ve exclusion of “outlier” observa)ons
• Try different analyses/models un)l get p<0.05
2016-10-17 | UC Berkeley
Alasdair Cohen | Lecture for Publich Health 250B
10
Source: Leif D. Nelson, UCB,
2016-06 BITSS presenta)on
”False-posi)ves, p-hacking,
sta)s)cal power, and eviden)al
value”
