Leading opinionNegative results: why do they need to be published?Peter SandercockThis short narrative review article defin...
P. Sandercock                                                                                  Leading opinionimportant di...
Upcoming SlideShare
Loading in...5

Sandercock, 2011 -_negative_trials_(int_j_stroke)_pdf


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Transcript of "Sandercock, 2011 -_negative_trials_(int_j_stroke)_pdf"

  1. 1. Leading opinionNegative results: why do they need to be published?Peter SandercockThis short narrative review article defines ‘negative results’ There are many scientific reasons for publishing negative orand cites several ethical and scientific reasons why such neutral (i.e. uninformative) studies; chiefly, they contain valu-studies should be made publicly available. able information, which should become part of the scientific record on the subject under study. Systematic reviews are anKey words: clinical trial, methodology, publication bias, stroke essential part of the research cycle (5). When scientists planunits, systematic reviews, treatment new research studies (clinical trials or observational studies in humans or experiments on laboratory animals), the first stepWhat do I mean by ‘negative results’? The term applies to should be a systematic review of the evidence (5). Such astudies conducted both in human and in animal subjects and review may reveal that the question has already been answeredencompasses three different types of result: reliably, or it may indicate that a further study is justified. For• truly inconclusive with ‘no evidence of effect’, generally example, the UK Medical Research Council and the UK Healthbecause the study was too small and inadequately powered Technology Appraisal Programme require that new applica-(several of the small studies included in the Cochrane system- tions for clinical trial funding should have performed (or atatic review of stroke units are in this category) (1); least cite) an up-to-date systematic review of the subject to• a well-conducted study, which is sufficiently large to provide ensure that the new research really is justified. If during the‘clear evidence of no effect’, i.e. that any effect is too small to be course of a clinical study, a large negative or neutral study isworthwhile pursuing either clinically or in further research published, this might require the trial steering committee or(the Clots in Legs Or Stockings after Stroke (CLOTS) trial data monitoring committee to pause for thought and considerof graded compression stocking for deep vein thrombosis whether the study should continue or be modified in someprevention is a good example) (2); or way. At the end of any clinical study, the results should pref-• clear evidence of harm when benefit had been expected. erably be presented in context of all the available evidence. Unfortunately, many such ‘negative’ yet still important For clinical trials, this is a requirement of the Consolidatedstudies in man (3) and in animals (4) remain unpublished. Standards of Reporting Trials (CONSORT) guidelines on Why should negative studies be published? The most publication of randomized clinical trials (6).important reason is ethics. If human subjects have given Systematic reviews can put a small but strikingly positiveconsent to participate in a clinical research study, be it a treat- study into context. For example, when a strikingly positivement trial, or an observational study, they have done so in the small study is viewed within the totality of the evidence avail-clear understanding that the research results will in some way able, it becomes evident that it is a freak ‘lucky’ result arisingbe of benefit to other people and contribute to scientific from the play of chance, and not a reliable estimate of the trueadvance. Furthermore, these human subjects have exposed effect (4,7). The availability of the ‘negative’ studies thenthemselves to risk and inconvenience by participating in the ensures that the one small positive (but outlying) study isstudy, and the justification for doing that ‘good deed’ should interpreted appropriately. There are numerous examples frombe that the author makes the data publicly available and the literature where small clinical trials or small studies ofensures that it is put to good use. Although animals do not give genetic associations produce striking positive results (oftenconsent to participate in research, we still have an ethical duty leading to a high-profile paper in a major journal), which areto make the best use of the data from any animals used in such then not subsequently replicated when larger, more reliableresearch. (neutral or negative) studies are published (8–10). To be reliable, systematic reviews need to include all rel-Correspondence: Peter Sandercock, Division of Clinical Neurosciences, evant randomized trials. The Stroke Unit Trialists’ Collabora-University of Edinburgh, Western General Hospital, Bramwell Dott tive systematic review of organized inpatient stroke care (1)Building, Edinburgh EH4 2XU, UK.Email: peter.sandercock@ed.ac.uk very clearly demonstrates the importance of making negativeTwitter: @IST_3 trial results available for inclusion in systematic reviews. Four- teen of the 16 trials comparing a stroke unit with generalConflict of interest: None declared. medical ward care were ‘negative’ for the effect on death (theyDOI: 10.1111/j.1747-4949.2011.00723.x were all underpowered to detect a moderate but clinically © 2011 The Author.32 International Journal of Stroke © 2011 World Stroke Organization Vol 7, January 2012, 32–33
  2. 2. P. Sandercock Leading opinionimportant difference), and four were never published. of Edinburgh Datashare site is an example http://datashare.However, the overall estimate of the effect from all the trials is.ed.ac.uk/).together is that stroke unit care reduces the odds of death by In summary, if a study involves informed consent in17% (95% confidence interval 4–29%) (P = 0·01), a result humans or the use of whole animals, and has been satisfacto-which has helped support the introduction of stroke unit care rily conducted, it should appear in the publicly available sci-into practice worldwide. What would have happened if all of entific record, irrespective of its overall conclusions.the apparently ‘negative’ studies had remained unpublished?We might well have ‘lost’ one of the most significantly benefi-cial interventions for stroke! Acknowledgements There are many obstacles to publication of ‘neutral or nega- I would like to thank William Whiteley, Charles Warlow, andtive’ studies; the first is the author. A study that does not have Norberto Cabral for helpful comments.a big fat juicy P-value does not stir the author’s adrenalinsufficiently to write the paper in the first place. It may not bein the commercial interest of the sponsors of negative research Referencesfor the results to be published. Journals like clear stories with 1 Stroke Unit Trialists’ Collaboration. Organised inpatient (stroke unit)definite results to increase their sales. Negative studies do not care for stroke. Cochrane Database of Systematic Reviews 2007; Issue 4.do much for journal income! There have been reports that Art. No.: CD000197. DOI: 10.1002/14651858.CD000197.pub2.some journals would only publish results from studies that are 2 Dennis M, Sandercock PA, Reid J et al. Effectiveness of thigh-lengthstatistically significant at the P < 0·05 level. This is clearly graduated compression stockings to reduce the risk of deep veinabsurd, but I suspect that the practice still continues (perhaps thrombosis after stroke (CLOTS trial 1): a multicentre, randomised controlled trial. Lancet 2009; 373:1958–65.rather more covertly these days). In the fast-moving world of 3 Gibson L, Brazzelli M, Thomas B, Sandercock P. A systematic reviewthe Internet and social media, where only the most strikingly of clinical trials of pharmacological interventions for acute ischaemicpositive results gain their ‘15 minutes of fame’, it is difficult to stroke (1955–2008) that were completed, but not published in full.ensure that neutral or negative results from well-conducted Trials 2010; 11:43. http://www.trialsjournal.com/content/11/1/43research would reach the public domain. 4 Sena ES, van der Worp HB, Bath PMW, Howells DW, Macleod MR. Publication bias in reports of animal stroke studies leads to major There are a number of possible solutions. Journal editors overstatement of efficacy. PLoS Biol 2010; 8:e1000344.could (perhaps) be persuaded to prioritize publishing well- 5 Bath PM, Gray LJ. Systematic reviews as a tool for planning andconducted negative research over poorly conducted positive interpreting trials. Int J Stroke 2009; 4:23–7.research. There are easier alternatives: although the Journal 6 Moher D, Hopewell S, Schulz KF et al. CONSORT 2010 explanationof Negative Results in Biomedicine (http://www.jnrbm.com/) and elaboration: updated guidelines for reporting parallel group ran- domised trials. BMJ 2010; 340:c869.provides a rather specific destination, there are now many 7 Collins R, MacMahon S. Reliable assessment of the effects of treatmentopen-access publishing journals. Furthermore, several grant- on mortality and major morbidity, I: clinical trials. [Review] [74 refs].giving bodies, such as the UK Medical Research Council and Lancet 2001; 357:373–80.the Wellcome Trust, expect data from research they have 8 Ioannidis JPA. Contradicted and initially stronger effects in highlyfunded to be published in an open-access format (applications cited clinical research. JAMA 2005; 294:218–28. 9 Ioannidis JP. Why most discovered true associations are inflated.for research grants now need to incorporate anticipated costs Epidemiology 2008; 19:640–8.for such publications). For authors without such funding, 10 Ioannidis JPA, Panagiotou OA. Comparison of effect sizes associatedmany institutions now make datasets accumulated by their with biomarkers reported in highly cited individual articles and inscientists freely available via open data repositories (University subsequent meta-analyses. JAMA 2011; 305:2200–10.© 2011 The Author.International Journal of Stroke © 2011 World Stroke Organization Vol 7, January 2012, 32–33 33