C O P E  Washington 9 Nov09
Upcoming SlideShare
Loading in...5
×
 

C O P E Washington 9 Nov09

on

  • 702 views

committee on publication ethics talk

committee on publication ethics talk
research misconduct
role of editors
research reporting guidelines
conflict of interest checklists for editors

Statistics

Views

Total Views
702
Views on SlideShare
702
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • These meanings reflect current popular usage and I’ll use them here. Dickison makes these distinctions.
  • The bias is almost always in favour of the positive outcome. There is a growing literature documenting the effect size of these biases. The effect sizes are substantial. Dickinson has a nice bibliography for those interested in a particular bias.
  • Chan et al looked at studies funded by the Canadian Institutes of Health Research during the late 1990’s and then tried to find published versions of these studies. They compared the funded protocol with the published papers. Despite careful literature searches and contacts with the funded researchers they were unable to find publications for 40% of the funded trials. (unreported). Of the reported trials between 31% (for efficacy outcomes) and 59% (for harm outcomes) were incompletely reported.
  • In a study with several outcome measures - key results, which ones get published? This is clearest for RCTs. An RCT must specify a primary hypothesis before the study begins. This is called the primary outcome and is the main outcome that must be reported. Sample size is usually calculated on the basis of this outcome. Observational studies may also have prespecified hypotheses. These are clearly described in study protocols. Published reports must report the results related to the primary hypthoses. To do otherwise - say to look at all the results and associations and only present those that are positive is misleading to readers: An observational study then becomes a data mining expedition - or fishing expedition. (That’s ok, but it ought to be reported as such - as an exploratory study. Let me give a personal expample. Tell me if it is malfeacance or incompetence? This is a published study in CMAJ. It was an RCT of an intensive intervention for patient with adult onset diabetes mellitus to determine if the intervention led a better outcome for the patients. This is an important question. Patients with diabetes are very difficult to manage, frequently do not follow advice and the disease carries a high risk of serious and fatal events if not properly managed. Thus there is great interest in seeing whether we can improve this situation. Would more intensive management - more frequent tests, diet counselling, nurse specialists encouraging and monitoring patients and so on improve outcomes? The protocol specified several outcomes that were of interest. Indeed a lot of outcomes - the differences between the patients in the control and experimental groups at the end of the study - were available to the authors. In the manuscript submitted, the Abstract (and the discussion of the paper) omitted the sentence that is unreadable on this slide. Thus there conclusion was that intensive therapy was “successful” blah blah blah. The result was positive. But this was not the key result of interest specified as the primary objective of the study. The primary objective was to compare a specific objective after 6 months of therapy had stopped. For that result there was no benefit and this was a negative trial according to our definitions. This was a publicly funded trial. So lets look at RCTs. Do published RCTs report the primary prespecified outcome(s)? One of the more interesting studies of this is work by AnWen Chen and Doug Altman.
  • In a study with several outcome measures - key results, which ones get published? This is clearest for RCTs. An RCT must specify a primary hypothesis before the study begins. This is called the primary outcome and is the main outcome that must be reported. Sample size is usually calculated on the basis of this outcome. Observational studies may also have prespecified hypotheses. These are clearly described in study protocols. Published reports must report the results related to the primary hypthoses. To do otherwise - say to look at all the results and associations and only present those that are positive is misleading to readers: An observational study then becomes a data mining expedition - or fishing expedition. (That’s ok, but it ought to be reported as such - as an exploratory study. Let me give a personal expample. Tell me if it is malfeacance or incompetence? This is a published study in CMAJ. It was an RCT of an intensive intervention for patient with adult onset diabetes mellitus to determine if the intervention led a better outcome for the patients. This is an important question. Patients with diabetes are very difficult to manage, frequently do not follow advice and the disease carries a high risk of serious and fatal events if not properly managed. Thus there is great interest in seeing whether we can improve this situation. Would more intensive management - more frequent tests, diet counselling, nurse specialists encouraging and monitoring patients and so on improve outcomes? The protocol specified several outcomes that were of interest. Indeed a lot of outcomes - the differences between the patients in the control and experimental groups at the end of the study - were available to the authors. In the manuscript submitted, the Abstract (and the discussion of the paper) omitted the sentence that is unreadable on this slide. Thus there conclusion was that intensive therapy was “successful” blah blah blah. The result was positive. But this was not the key result of interest specified as the primary objective of the study. The primary objective was to compare a specific objective after 6 months of therapy had stopped. For that result there was no benefit and this was a negative trial according to our definitions. This was a publicly funded trial. So lets look at RCTs. Do published RCTs report the primary prespecified outcome(s)? One of the more interesting studies of this is work by AnWen Chen and Doug Altman.
  • Approved by US FDA 1994 RX partial seizures By 2003, one of Pfizer’s best selling drugs for minor seizures. Off-label uses account for 90% of sales migraine, bipolar disorders, OCD, depression, insommnia, etc.. Adverse effects - dizziness, mood swings etc.. hepatotoxcity, depression, suicide, Court cases - for illegally marketing a drug based on no evidence of efficacy
  • Dickinson’s analysis of another paper that was used by Pfizer to promote off-label uses of Neurontin. Clearly this is a seriously flawed RCT, as published by an editor, I hope not one in this room.
  • Does it matter. Well yes. This misleadingly reported study is still being cited, the drug is still being recommended for prevention of migraine and worse, has found it’s way into guidelines that are Evidence Based, whatever that means in this context.
  • Group 2 are highly recommended based on RCT evidence.

C O P E  Washington 9 Nov09 C O P E Washington 9 Nov09 Presentation Transcript

  • Published Research Flawed, misleading, deceitful ? John Hoey COPE U.S Seminar 2009 Washington, DC [email_address] www.slideshare.com/hoey
    • Role’s, Responsibilities of Editors
    • Research manuscripts
      • Bailar’s pyramid of faults
        • Examples
    • What can an editor do?
      • Reporting guidelines
      • Conflict of Interest checklists
    Outline
  • COPE Code of Conduct General duties and responsibilities of Editors Be responsible for everything published in their journals. • Strive to meet the needs of readers and authors • constantly improve the journal • Ensure the quality of the material they publish • champion freedom of expression • Maintain the integrity of the academic record • Preclude business needs from compromising intellectual standards • always be willing to publish corrections, clarifications, retractions and apologies when needed.
  • Maintaining the Integrity of the Scientific Record.......why?
    • Ethics - Nuremburg Trials - Helsinki Declaration.
    • Harm to patients and the public
      • Physical harm
      • Financial harm
  • Editorial Ethical Responsibilites
    • Helsinki Declarations
    “ Both authors and publishers have ethical obligations. In publication of the results of research, the investigators are obliged to preserve the accuracy of the results. Negative as well as positive results should be published or otherwise publicly available. Sources of funding, institutional affiliations and any possible conflicts of interest should be declared in the publication. Reports of experimentation not in accordance with the principles laid down in this Declaration should not be accepted for publication.”
  • $$ Conflict of Interest Carelessness Incompetence John Bailar’s Pyramid of Manuscript Problems
  • a few examples of problems editors face
  • Reporting Bias Kay Dickinson, Reporting and other biases in studies of Neurontin for migraine, psychiatric/bipolar disorders, nociceptive pain, and neuropathic pain . August, 2008 http://dida.library.ucsf.edu/pdf/oxx18r10
  • Some definitions
    • Positive= Study shows results favourable to the interests of the author/sponsor. Usually statistically significant.
    • Negative = Study shows results unfavourable to the interest of the the author/sponsor - may or may not be statistically significant.
  • Reporting Biases
    • Non-publication of negative or neutral results
    • Selective publication of results - outcome bias
    • Multiple publication bias
    • Language bias - and publishing in the grey literature
    • Time lag bias
    • Undeclared conflicts of interest
    • Ghost writing
    Dickinson
  • Hypothesis testing or Hypothesis generating? Hypothesis testing Hypothesis generating Prior specification before study begins eg. RCT, observational studies, etc. Finding an interesting result among man possible results eg. Survey, cohort study etc..
  • Selective publication - Outcome bias publishing the more interesting (usually positive) result Was there an hypothesis? A plan for analysis and reporting of data? In an RCT, this is the primary outcome
  • RCT R Rx A Placebo Outcome
  • Study design - RCT R Rx A Placebo Outcome Primary Outcome -Specified in Protocol? or fishing expedition?
  • Consort Statement
    • Poor quality of reporting RCTs.
      • Could not be trusted for use in clinical practice
      • Could not be combined in systematic reviews
    • Identified 22 essential items needed to conduct a RCT.
    Ann Intern Med. 2001;134:663-694. www.annals.org David Moher
  • Selective publication - Outcome bias (publishing the more interesting result) 1402 outcomes 31% - 59% incompletely reported (40% not reported at all) Chan, A.-W. et al. CMAJ 2004;171:735-740 48 RCTs funded by national granting agency
  • Selective publication - Outcome bias (publishing the more interesting result) Primary Outcome Diabetic control 6 months after the end of intensive multithearpy
  • Selective publication - Outcome bias (publishing the more interesting result) Interpretation: Intensive multitherapy for patients with poorly controlled type 2 diabetes is successful in helping patients meet most of the goals set by a national diabetes association. However, 6 months after intensive therapy stopped and patients returned to usual care the benefits had vanished, However, 6 months after intensive therapy stopped and patients returned to usual care the benefits had vanished.
  • does any of this matter?
  • Neurontin (gabapentin) images from Wikipedia which also has a nice summary of court proceedings and results For minor seizures FDA approved 1994 By 2003 one of Pfizer’s best selling drugs Off-label uses account for 90% of sales Multiple small RCTs Benefit for other disorders e.g. migraine
  • P Wessely, C Baumgartner, D Klinger, J Kreczi, N … - Cephalalgia, 1987 A seriously flawed RCT - Accepted for publication by someone Bias Example Publication Final negative primary results not published, only positive preliminary results Selective outcome reporting Outcome reported was not primary or secondary outcome Selective statistical analyses 2 nonrandomized patients assigned to neurotin were include with those randomized Spin Emphasis on “positive” outcomes
  • 16 Citations P Wessely , C Baumgartner, D Klinger, J Kreczi, N … - Cephalalgia, 1987 Does it matter? General Principles of Migraine Management: The Changing Role of Prevention E Loder, D Biondi - Headache: The Journal of Head and Face Pain, 2005 - Blackwell Synergy Preventive treatment of migraine - SD Silberstein - Trends in Pharmacological Sciences, 2006 - Elsevier Migraine prevention DW Dodick, SD Silberstein - British Medical Journal, 2007 - pn.bmj.com Neuromodulators for Migraine Prevention R Kaniecki - Headache: The Journal of Head and Face Pain, 2008 - Blackwell Synergy Practice parameter: Evidence-based guidelines for migraine headache (an evidence-based review) Stephen D. Silberstein , MD, FACP, for the US Headache Consortium * Neurology 2000;55:754-762
  •  
  •  
  • a typical editor and their support staff... the editor’s desk
  • BMJ Editorial Staff Editor in chief Fiona Godlee email T: + 44(0)20 7383 6102 Deputy editors Jane Smith email T:+44 (0)20 7383 6009 Tony Delamothe email T: +44 (0) 20 7383 6006 Trish Groves email T:+44 (0)20 7383 6018 Magazine editor Trevor Jackson email T:+44 (0)20 7383 6677 bmj.com editor David Payne email T:+44 (0)20 7383 6532 Editorials editor Giselle Jones Research papers editors Kristina Fister email Trish Groves email Elizabeth Loder email Christopher Martyn email Alison Tonks email Primary care editor Domhnall MacAuley email News editors Annabel Ferriman email T: 44 (0)20 7383 6035 Zosia Kmietowicz email Features editors Deborah Cohen email T: +44 (0)20 7383 6183 Rebecca Coombes email T: +44 (0)20 7383 6243 Clinical reviews editor Kirsten Patrick email Practice editor Mabel Chew email Analysis editor Tessa Richards email T: +44 (0) 20 7383 61 Letters and obituaries editor Sharon Davies email T:+44 (0)20 7383 6716 Observations and reviews editor Trevor Jackson email T:+44 (0)20 7383 6677 Roger Robinson editorial registrar Helen Macdonald email T: + 44 (0)20 7 874 7022 Senior researcher Sara Schroter email T: +44 (0)20 7383 6744 Patient editor Peter Lapsley email Web team Editor bmj.com David Payne email Assistant editor, bmj.com Birte Twisselmann email T: +44 (0)20 7383 6720 Print journal team Magazine editor Trevor Jackson email Designer Jane Walker email Senior art worker Adam di Chiara email Picture editor Vanessa Fletcher email Deputy managing editor Lucy Banham email T:+44 (0)20 7383 6365 Technical editors Jackie Annis email T:+44 (0)20 7383 6658 Maggie Butler email T:+44 (0)20 7383 6074 Sally Carter email T:+44 (0)20 7383 6659 Margaret Cooter email T:+44 (0)20 7383 6657 Greg Cotton email T:+44 (0)20 7383 6685 Clare Griffith email T: +44 (0)20 7383 6051 Richard Hurley email T: + 44 (0)20 7383 6051 Elizabeth Payne email T: + 44 (0)20 7383 6449 Karl Sharrock email T:+44 (0)20 7383 6658 Barbara Squire email T:+44 (0)20 7383 6658 Julia Thompson email T:+44 (0)20 7383 6691 Douglas Kamerow email PA to editor in chief Julia Burrell email T:+44 (0)20 7383 6102 Departmental administrator Chelsey White email T:+44 (0)20 7383 6109 Benchpress database manager Gary Bryan email T:+44 (0)20 7383 6304 Benchpress administrator Sue Minns email T:+44 (0)20 7383 6309 Chief production editor John Mayor email T:+44 (0)20 7383 6355 Assistant production editor Edwyn Mayhew email T:+44 (0)20 7383 6145 Production editor Jett Aislabie email T:+44 (0)20 7874 7014 Malcolm Brown (maternity cover) T:+44 (0)20 7874 7014 Illustrator Anthea Wilkie email T:+44 (0)1737 215143 Copyright administrator email Career Focus Edward Davies email T: + 44 (0) 20 7383 6562 studentBMJ Jessie Colquhoun [email_address] T: +44 (0)20 7874 7016 Visiting editors Jennifer Leaning (USA) Ray Moynihan (USA) Joanne Roberts (USA) Charlie Wilson (USA) Editorial advisers Steven Reid Ian Maconochie Peter Leman Nick Dunn Frank Sullivan Pippa Oakeshott Aziz Sheikh Lucy Chappell Christopher Whitty Scott Murray Julia Hippisley-Cox Josip Car Robin Fox Sue Morgan Statistical advisers Doug Altman Tim Cole Hazel Inskip Julie Morris Deborah Ashby Jon Deek
  • The Lancet Editorial office “ The Apartment” Billy Wilder, Jack Lemnon, 1959
  • Editor David Bevan MB Toronto
  • Photo by Waldo www.flickr.com/photos/waldo4/2178788631/ Manuscripts are messy & confusing
  • Editorial checklists for authors
  • RCTs Consort guideline Checklists for KEY elements of a study that need to be reported in published papers. Minimal required content www.equator-network.org
  • ? CONSORT Enrolment Allocation Follow-up Analysis
  • Editorial checklists for authors
    • Guidelines for research reporting
    • Consort
    • Prisma
    • Others ~ 60 known guidelines
    www.equator-network.org
  • Editorial checklists for authors www.equator-network.org
    • Notions
    • Key elements of research that ought to be reported
    • For use by
      • authors in writing manuscripts
      • researchers writing grants
      • grant review committees
      • EDITORS
  • What can editor’s do to improve quality of what they publish?
    • Explicit contract
      • Helsinki Declaration
      • WAME/ICMJE
      • Publisher goals
    • Disclosure of contract
    • Disclosure of editorial & publisher Conflicts of interest
    Publisher/Editor Author/Editor
    • Aim for quality
    • Publish less
    • Author guidelines/instructions
    • Use reporting guidelines
      • Require authors use them
      • Use checklists with submissions
  • Conflict of Interest -
  • Conflict of Interest -
  • Conflict of Interests - Who was responsible for?
    • Study
        • design
        • data collection
        • analysis
        • write up
        • decision to publish
        • etc
    • 1109 Canadian clinicaltrials.gov
    • 732 investigators
    • 11 ideal practices to mitigate fCOI
      • - control over design
      • - data collection
      • - analysis
      • - interpretation
      • -write up
      • -authorship
      • -decision to publish
      • -etc.
    • Only 6 % of investigators met all ideal practices
    Rochon P, et al. - manuscript in review
  •  
  • Thank-you www.equator-network.org [email_address] www.slideshare.com/hoey