Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
“Evidenced-Based” Behavioral
Medicine as Bad as
Bad Pharma
James C. Coyne, Ph.D.
University Medical Center
Groningen (UMCG...
Introduction to ICBM Symposium
Groningen, August 2014
Are findings in behavioral medicine believable?
Nick Brown. Physical...
John Ioannidis documented many positive
findings in the biomedical literature do not
replicate and many apparent “discover...
“It can be proven that most claimed
research findings are false”
Ioannidis, JPA. (2005). Why most published
research findi...
Ben Goldacre
Drugs are tested by the people who
manufacture them, in poorly designed
trials, on hopelessly small numbers o...
Efforts at reform
 Preregistration of clinical trials make it more
difficult to hide negative trials or alter analytic
pl...
Our modest contribution to
reform
Roseman, M, Milette, K, Bero, LA, Coyne, JC,
Lexchin, J., Turner, EH, & Thombs, BD. (201...
We documented with association of source of
funding with claimed size of effects.
We proposed considering conflict of inte...
Cochrane Risk of Bias
 Sequence generation.
 Allocation concealment.
 Blinding of participants, investigators, outcome
...
In 1979 he wrote, "It is surely a great criticism of our
profession that we have not organised a critical
summary, by spec...
Clinical psychology and
behavioral medicine
CONSORT adopted later and less consistently
than Biomedicine.*
Preregistration...
Trickle down of reform
 Changes forced upon Pharma slowly and
inconsistently reach clinical psychology.
 Larger data set...
Clinical psychology and
behavioral medicine
RCT literature dominated by methodologically flawed,
underpowered studies obta...
 Evidence-Based Medicine developed to weed
out ineffective treatments.
 Evidence-Based Behavioral Medicine
developed to ...
The Two Tribes of EBBM:
Dodoists and Skeptical Dogs
Beware of Professional
Organization Involvement as a Risk
of Bias
Guidelines for Guidelines
 Shaneyfelt T. In guidelines we cannot trust. Arch Intern
Med;172:1633-1634.
 Institute of Med...
AHA Advisory on Screening for
Depression Not Guideline-Congruent!
 Ziegelstein RC, Brett D, Thombs BD, Coyne JC, de
Jonge...
Beware “Official” meta-analyses
(Critiqued here)
 Coyne JC, Thombs BD, Hagedoorn M. Ain’t Necessarily
So: Review and Crit...
SBM Initiative
Meta-analyses generated by professional
organizations should receive special
critical scrutiny because of t...
“We are grateful to the Society of Behavioral
Medicine (SBM) for selecting the authorship
group. This article is one of th...
What to watch for in meta analyses
commissioned by professional
organizations
 Precommitment to conclusion that intervent...
How This Symposium Came About
EHPS 2013: Improving the Credibility of
Health Psychology Intervention Research:
Problems and Solutions
Ozakinci, G., Whit...
Picking up collaborators on
the Internet
Nick Brown
Moritz Heene
Daniel Lakens
“Evidenced based” behavioral medicine as bad as bad pharma
“Evidenced based” behavioral medicine as bad as bad pharma
Upcoming SlideShare
Loading in …5
×

“Evidenced based” behavioral medicine as bad as bad pharma

2,717 views

Published on

Introduction to symposium held at International Congress of Behavioral Medicine, Groningen, August 2014. Discusses the shortcomings of evidence-based behavioral medicine in light of efforts to reform the shortcomings of the Pharma literature.

Published in: Science
  • This comes as no surprise to anyone has watched a beloved son suffering from schizophrenia who was rational enough to feel the helplessness and depression knowing that his treatment for his chronic brain disease was in vain. Nothing quelled the threatening voices that threatened him constantly. It must have seemed easier to him to kill himself by thrusting two knives into his eyes until they pierced his brain than to live with his disease any longer.

    And to think his care-givers had openly discussed with him the possibility that I had caused his disease!

    It will take scientific methodology to find the biological agents that infect human brains. We must no longer accept that social studies can bring viable answers for treatment of mental illness system. Even its victims now know better than to accept it as treatment with counselling. It will become medical malpractice to base diagnosis and treatment on the shifting sands of sociology.

    But I want to thank those professionals who are willing to shout from the rooftops that for decades society got it so wrong.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

“Evidenced based” behavioral medicine as bad as bad pharma

  1. 1. “Evidenced-Based” Behavioral Medicine as Bad as Bad Pharma James C. Coyne, Ph.D. University Medical Center Groningen (UMCG), Groningen, the Netherlands jcoynester@gmail.com
  2. 2. Introduction to ICBM Symposium Groningen, August 2014 Are findings in behavioral medicine believable? Nick Brown. Physical health outcomes in positive psychology: Weak evidence for strong claims. James C. Coyne, Moritz Heene, Gozde Ozakinci. Unsafe Dependence of Meta Analyses in Behavioral Medicine on Failsafe N. Daniel Lakens. How likely is it an intervention study will replicate? The pain literature as an example.
  3. 3. John Ioannidis documented many positive findings in the biomedical literature do not replicate and many apparent “discoveries” turn out to be exaggerated or simply false. Many apparent discoveries are created by a combination of confirmatory bias, flexible rules of design, data analysis and reporting and significance chasing.
  4. 4. “It can be proven that most claimed research findings are false” Ioannidis, JPA. (2005). Why most published research findings are false. PLOS Medicine 2: 696-701. Ioannidis, JPA. (2005). Contradicted and initially stronger effects in highly cited clinical research. JAMA 294: 218-228. Young, NS., Ioannidis, JPA. et al. (2008). Why Current Publication Practices May Distort Science. PLOS Medicine 5: 1418-1422.
  5. 5. Ben Goldacre Drugs are tested by the people who manufacture them, in poorly designed trials, on hopelessly small numbers of weird, unrepresentative patients, and analysed using techniques which are flawed by design, in such a way that they exaggerate the benefits of treatments. Unsurprisingly, these trials tend to produce results that favour the manufacturer. When trials throw up results that companies don't like, they are perfectly entitled to hide them from doctors and patients.
  6. 6. Efforts at reform  Preregistration of clinical trials make it more difficult to hide negative trials or alter analytic plans after results are known.  Reporting standards ensure more transparent and detailed article so results can be independently validated.  Reporting standards and pre-registration of plans for meta-analyses and systematic reviews.  Making data available for independent reanalysis.
  7. 7. Our modest contribution to reform Roseman, M, Milette, K, Bero, LA, Coyne, JC, Lexchin, J., Turner, EH, & Thombs, BD. (2011). Reporting of conflicts of interest in meta- analyses of trials of pharmacological treatments. JAMA, 2011;305(10):1008-17. Roseman, M, Turner, EH, Lexchin, J., Coyne, JC, Bero, LA, & Thombs, BD. (2012). Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study. BMJ, 2012; 345.
  8. 8. We documented with association of source of funding with claimed size of effects. We proposed considering conflict of interest as a moderator, source of heterogeneity, and another risk of bias in Cochrane risk of bias assessment. The Cochrane Collaboration agreed.
  9. 9. Cochrane Risk of Bias  Sequence generation.  Allocation concealment.  Blinding of participants, investigators, outcome assessors.  Incomplete outcome data.  Selective outcome reporting.  Other threats to validity.
  10. 10. In 1979 he wrote, "It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials” Prof Archibald Cochrane, CBE (1909 - 1988) The Cochrane Collaboration is named in honor of Archie Cochrane, a British researcher.
  11. 11. Clinical psychology and behavioral medicine CONSORT adopted later and less consistently than Biomedicine.* Preregistration of trials is now encouraged, but enforcement is lax. Conflict of interest less acknowledged, although investigator allegiance pervasive and a risk of bias. *APS (Association of Psychological Science) journals not yet compliant.
  12. 12. Trickle down of reform  Changes forced upon Pharma slowly and inconsistently reach clinical psychology.  Larger data sets allow exploration of issues such as choice of control groups and investigator allegiance.  Behavioral Medicine adopts changes occurring in clinical psychology later and inconsistently.
  13. 13. Clinical psychology and behavioral medicine RCT literature dominated by methodologically flawed, underpowered studies obtaining significant results at statistically improbable rate. Weak control groups, unusual to have active control group. Flexible rules of design and selective reporting of outcomes chosens after results are known. Strong investigator allegiance effects.
  14. 14.  Evidence-Based Medicine developed to weed out ineffective treatments.  Evidence-Based Behavioral Medicine developed to demonstrate that treatments worked and should be disseminated and reimbursed.
  15. 15. The Two Tribes of EBBM: Dodoists and Skeptical Dogs
  16. 16. Beware of Professional Organization Involvement as a Risk of Bias
  17. 17. Guidelines for Guidelines  Shaneyfelt T. In guidelines we cannot trust. Arch Intern Med;172:1633-1634.  Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines (2011). Clinical practice guidelines we can trust. Washington, DC: National Academies Press.  Loblaw, D. A., Prestrud, A. A., Somerfield, M. R., et al. (2012). American Society of Clinical Oncology clinical practice guidelines: Formal systematic review–based consensus methodology. J Clin Oncol, 30(25), 3136-3140.
  18. 18. AHA Advisory on Screening for Depression Not Guideline-Congruent!  Ziegelstein RC, Brett D, Thombs BD, Coyne JC, de Jonge P. Routine Screening for Depression in Patients with Coronary Heart Disease: Never Mind. Journal of the American Academy of Cardiology. 2009;54(10):886-90.  Thombs, B. D., Jewett, L. R., Knafo, R., Coyne, J. C., & Ziegelstein, R. C. (2009). Learning from history: a commentary on the American Heart Association Science Advisory on depression screening. American heart journal, 158(4), 503.
  19. 19. Beware “Official” meta-analyses (Critiqued here)  Coyne JC, Thombs BD, Hagedoorn M. Ain’t Necessarily So: Review and Critique of Recent Meta-Analyses of Behavioral Medicine Interventions in Health Psychology. Health Psychology. 2010;29(2):107-16.  Coyne, J. C. (2012). Re: Meta-analysis of Efficacy of Interventions Elevated Depressive Symptoms in Adults Diagnosed With Cancer. Journal of the National Cancer Institute, djs408.
  20. 20. SBM Initiative Meta-analyses generated by professional organizations should receive special critical scrutiny because of tendency to gloss over limits of literature in order to promote the services of their membership.
  21. 21. “We are grateful to the Society of Behavioral Medicine (SBM) for selecting the authorship group. This article is one of three meta- analyses that have been undertaken under the aegis of the SBM Evidence-Based Behavioral Medicine Committee; the other two meta- analyses examine the effects of psychosocial interventions on depression and fatigue among patients with cancer.”
  22. 22. What to watch for in meta analyses commissioned by professional organizations  Precommitment to conclusion that interventions work and are ready for dissemination and reimbursement.  Ignoring of preponderance of methodologically flawed, unpowered sources of bias.  Broad inclusion of diverse interventions into one category.  Ignoring statistical heterogeneity.  Weak assessment of risk of bias (Cochrane criteria are seldom used).
  23. 23. How This Symposium Came About
  24. 24. EHPS 2013: Improving the Credibility of Health Psychology Intervention Research: Problems and Solutions Ozakinci, G., Whitehead, R., Sobota, A., Coyne, JC. Inaccurate Abstracts in Health Psychology: The Problem and an Easily Implementable Solution. Coyne, JC. Too good to be true: Health psychology depends too much on positive underpowered studies.
  25. 25. Picking up collaborators on the Internet
  26. 26. Nick Brown
  27. 27. Moritz Heene
  28. 28. Daniel Lakens

×