Biostatistics and Ethics Shelley Hurwitz, PhD Brigham and Women’s Hospital Harvard Medical School Fellow, American Statistical Association Advisory Board on Ethics, International Statistical Institute

1. Biostatistics and Ethics Shelley Hurwitz, PhD Brigham and Women’s Hospital Harvard Medical School Fellow, American Statistical Association Advisory Board on Ethics, International Statistical Institute MedicReS World Congress on Good Medical Research New York October 17, 2014 1

2. Outline
•
Reputation of statistics
•
Statisticians respond
•
Ethical guidelines
•
Reproducibility 2

3. •
“Figures will not lie, but liars will figure.”
•
“There are three kinds of lies: Lies, damned lies, and statistics"
•
“Facts are stubborn things, but statistics are pliable”
•“How to lie with statistics”
•"Statistics are no substitute for judgment.”
Reputation of Statistics 3

4. •
“Figures will not lie, but liars will figure.” (1889)
•
“There are three kinds of lies: Lies, damned lies, and statistics"
•
“Facts are stubborn things, but statistics are pliable”
•
“How to lie with statistics”
•
"Statistics are no substitute for judgment.”
Reputation of Statistics 4

5. •
“Figures will not lie, but liars will figure.”
•
“There are three kinds of lies: Lies, damned lies, and statistics” (1906)
•“Facts are stubborn things, but statistics are pliable”
•“How to lie with statistics”
•"Statistics are no substitute for judgment.”
Reputation of Statistics 5

6. •
“Figures will not lie, but liars will figure.”
•
“There are three kinds of lies: Lies, damned lies, and statistics"
•
“Facts are stubborn things, but statistics are pliable”
•“How to lie with statistics”
•"Statistics are no substitute for judgment.”
Reputation of Statistics 6

7. •
“Figures will not lie, but liars will figure.”
•
“There are three kinds of lies: Lies, damned lies, and statistics"
•
“Facts are stubborn things, but statistics are pliable”
•
“How to lie with statistics” (1954)
•
"Statistics are no substitute for judgment.”
Reputation of Statistics 7

8. •
“Figures will not lie, but liars will figure.”
•
“There are three kinds of lies: Lies, damned lies, and statistics"
•
“Facts are stubborn things, but statistics are pliable”
•
“How to lie with statistics”
•
"Statistics are no substitute for judgment.”
Reputation of Statistics 8

9. Outline
•
Reputation of statistics
•
Statisticians respond
•
Ethical guidelines
•
Reproducibility 9

10. Series of articles in British Medical Journal, 1980-1
“Statistics and Ethics in Medical Research”
by Douglas G. Altman
•Misuse of statistics is unethical
•Study design
•How large a sample?
•Collecting and screening data
•Analysing data
•Presentation of results
•Interpreting results
•Improving the quality of statistics in medical journals
10

11. John C Bailar III
"Science, Statistics, and Deception"
Annals of Internal Medicine 1986;104:259-60.
•Summarizes important role of statistics in scientific research
•Practices that can distort scientific inferences:
–Failure to deal honestly with readers about non-random error (bias)
–Post hoc hypothesis
–Inappropriate statistical tests and other statistical procedures
–Fragmentation of reports
–Low statistical power
–Suppressing, trimming, or “adjusting” data
–Undisclosed repetition of “unsatisfactory” experiments
–Selective reporting of findings 11

12. David L. DeMets
“Statistics and Ethics in Medical Research” Science and Engineering Ethics. 1999; 5(1) 97-117.
•Statistical methods are powerful tools but can easily be misused.
•Pressures to get “right” results can be difficult, especially for junior statisticians.
•Community must be vigilant; misuse is hard to detect.
•Just as no one may know if you disobey a stop sign late at night, rarely will anyone be able to double check in detail our statistical analyses and design.
•Ultimately, proper use of statistics in medical research rests with the integrity of the individual. 12

13. John S. Gardenier and David B. Resnik
“The Misuse of Statistics: Concepts, Tools, and a Research Agenda”
Accountability in Research: Policies and Quality Assurance. 2002 (9) 65-74.
•Misconduct
•Incompetence
•Negligence
•Honest error
•Deliberate deception
–Pressures to publish, produce results, or obtain grants
–Career ambitions or aspirations
–Conflicts of interest and economic motives
–Inadequate supervision, education, or training 13

14. Outline
•
Reputation of statistics
•
Statisticians respond
•
Guidelines for Ethical Statistical Practice
•
Reproducibility 14

15. The Statistical Society of Australia Inc.
Code of Conduct (1998)
American Statistical Association
Ethical Guidelines for Statistical Practice (1999)
Statistical Society of Canada
Code of Ethical Statistical Practice (2004)
International Statistical Institute
Declaration on Professional Ethics (2010)
Royal Statistical Society
Code of Conduct (2014) 15

16. American Statistical Association
Ethical Guidelines for Statistical Practice
1949: Commission on Statistical Standards to develop a code of ethical practices
1954: 2 surveys
1956: Interest was low. Idea was tabled.
1977: Interest increased, new committee formed.
1980: Interim Code of Conduct published, requesting comments. There was extensive input.
1983: Final version published.
1996: The Committee on Professional Ethics started drafting a revision.
1998: Draft published. Invited comments. Open session at annual meeting.
1999: Current version approved by Board of Directors. 16

17. Domains of Ethical Responsibility in ASA’s and other societies’ codes of conduct
•
Society
•
Public interest
•
The profession
•
Funders
•
Clients
•
Employers
•
Research subjects
•
Research team colleagues
•
Other statisticians 17

18. 15 years later New ethical concerns for statisticians
•
Massive amounts of data
•
Complex data structure
•
High dimensionality
•
De-identification
•
Security & encryption
•
Increasingly multidisciplinary studies 18

19. Outline
•
Reputation of statistics
•
Statisticians respond
•
Ethical guidelines
•
Reproducibility 19

20. “How Science Goes Wrong” The Economist, 10/19/2013
•
Half of published research cannot be replicated.
•
2000-10 roughly 80,000 patients took part in clinical trials based on research that was later retracted because of mistakes or improprieties
•
Lost opportunity, lost money
•
Obligation to publish or perish
•
Honest confusion “sweet signal of a genuine discovery” and “freak of the statistical noise”
•
Spurious correlations in newspapers, morning TV
•
Failure to report failure leads to more researchers exploring blind alleys already explored. 20

21. “Fraud in Medical Research: An International Survey of Biostatisticians” Ranstam, Buyse, George, Evans, Geller, Scherrer, Lesaffre, Murray, Edler, Hutton, Colton, and Lachenbruch, for the ISCB Subcommittee on Fraud Controlled Clinical Trials 21:415–427 (2000)
•
51% of biostatisticians know of at least one fraudulent project.
•
31% have been involved in fraudulent projects.
•
13% have been directly asked to support fraud.
–Fabrication or falsification of data
–Suppression or selective deletion of data
–Deceptive design or analysis
–Deceptive reporting of results 21

22. The Duke Saga
2006: Nature Medicine “Genomic signatures to guide the use of chemotherapeutics”
2009: The Annals of Applied Statistics
“Deriving chemosensitivity from cell lines: Forensic bioinformatics and reproducible research in high- throughput biology”
2011: Nature Medicine
“Retraction: Genomic signatures to guide the use of chemotherapeutics”
22

23. Nature Medicine 2006, Vol. 12, 1294 – 1300 “Genomic Signatures to Guide the use of Chemotherapeutics” By Anil Potti, Holly K Dressman, Andrea Bild, Richard F Riedel, Gina Chan, Robyn Sayer, Janiel Cragun, Hope Cottrill, Michael J Kelley, Rebecca Petersen, David Harpole, Jeffrey Marks, Andrew Berchuck, Geoffrey S Ginsburg, Phillip Febbo, Johnathan Lancaster & Joseph R Nevins 23

24. Corrigenda: Genomic signatures to guide the use of chemotherapeutics
By Anil Potti, Holly K Dressman, Andrea Bild, Richard F Riedel, Gina Chan, Robyn Sayer, Janiel Cragun, Hope Cottrill, Michael J Kelley, Rebecca Petersen, David Harpole, Jeffrey Marks, Andrew Berchuck, Geoffrey S Ginsburg, Phillip Febbo, Johnathan Lancaster & Joseph R Nevins
•
Nature Medicine 13, 1388 (2007)
•
Nature Medicine 14, 889 (2008) 24

25. The Annals of Applied Statistics 2009, Vol. 3, 1309 – 1334 “Deriving Chemosensitivity From Cell Lines: Forensic Bioinformatics and Reproducible Research in High-throughput Biology” By Keith A. Baggerly and Kevin R. Coombes 25

26. Nature Medicine 2011, Vol.17, 135 “Retraction: Genomic Signatures to Guide the use of Chemotherapeutics” By Anil Potti, Holly K Dressman, Andrea Bild, Richard F Riedel, Gina Chan, Robyn Sayer, Janiel Cragun, Hope Cottrill, Michael J Kelley, Rebecca Petersen, David Harpole, Jeffrey Marks, Andrew Berchuck, Geoffrey S Ginsburg, Phillip Febbo, Johnathan Lancaster & Joseph R Nevins 26

27. Uninteresting Work
•
Replication of positive results is not interesting work.
•
Negative studies are not that interesting.
•Open access to all data
•Clinical trial registration
•Journals should set aside room for
–Replication studies
–Negative studies
–Expanded methods sections
•Funders should set aside funding for this uninteresting work. 27

28. “Lies, Damned Lies, and Medical Science” by David H. Freedman October 4, 2010 The Atlantic 28

29. “Why Most Published Research Findings are False” by Professor John P. A. Ioannidis 2005, PLOS Medicine 1,138,461 on-line views 1,413 citations 10,464 shares on Facebook and Twitter 29

30. What reduces the likelihood a scientific finding is true?
1.
Studies conducted in the scientific field are small.
2.
Effect sizes in the field are small.
3.
Number of studies is high, pre-tested relationships are not selected for study.
4.
Highly flexible designs, definitions, outcomes, analyses.
5.
Greater financial and other interests and prejudices in a scientific field.
6.
Greater “hot-ness” of the field, with more teams involved.
Ioannidis, JPA (2005) “Why Most Published Research Findings are False”
30

31. Biostatistics and Ethics Conclusion
Updating Wright, 1889:
It is our duty, as practical statisticians, to prevent the investigator from figuring; in other words, to prevent him or her from perverting the truth, in the interest of some theory he or she wishes to establish.
DeMets, 1999:
Ultimately proper use of statistical methods must rest with the integrity of the individual statistician.
•Integrity
•Transparency
•Reproducibility
•Accountability
•Guidelines 31

32. Thank you
Contact information
hurwitz@hms.harvard.edu 32