The document discusses the credibility issues in behavioral medicine research, highlighting the prevalence of misleading findings due to methodological flaws, publication bias, and conflicts of interest. It emphasizes the importance of reforms such as preregistration of trials and improved reporting standards to enhance transparency and allow for independent validation of results. The author also critiques the tendency of professional organizations to produce biased meta-analyses that overlook the limitations of existing literature.

1. “Evidenced-Based” Behavioral
Medicine as Bad as
Bad Pharma
James C. Coyne, Ph.D.
University Medical Center
Groningen (UMCG), Groningen, the
Netherlands
jcoynester@gmail.com

2. Introduction to ICBM Symposium
Groningen, August 2014
Are findings in behavioral medicine believable?
Nick Brown. Physical health outcomes in positive
psychology: Weak evidence for strong claims.
James C. Coyne, Moritz Heene, Gozde Ozakinci.
Unsafe Dependence of Meta Analyses in Behavioral
Medicine on Failsafe N.
Daniel Lakens. How likely is it an intervention study will
replicate? The pain literature as an example.

3. John Ioannidis documented many positive
findings in the biomedical literature do not
replicate and many apparent “discoveries” turn
out to be exaggerated or simply false.
Many apparent discoveries are created by a
combination of confirmatory bias, flexible rules
of design, data analysis and reporting and
significance chasing.

4. “It can be proven that most claimed
research findings are false”
Ioannidis, JPA. (2005). Why most published
research findings are false. PLOS Medicine 2:
696-701.
Ioannidis, JPA. (2005). Contradicted and
initially stronger effects in highly cited clinical
research. JAMA 294: 218-228.
Young, NS., Ioannidis, JPA. et al. (2008). Why
Current Publication Practices May Distort
Science. PLOS Medicine 5: 1418-1422.

5. Ben Goldacre
Drugs are tested by the people who
manufacture them, in poorly designed
trials, on hopelessly small numbers of
weird, unrepresentative patients, and
analysed using techniques which are
flawed by design, in such a way that
they exaggerate the benefits of
treatments. Unsurprisingly, these
trials tend to produce results that
favour the manufacturer. When trials
throw up results that companies don't
like, they are perfectly entitled to hide
them from doctors and patients.

6. Efforts at reform
 Preregistration of clinical trials make it more
difficult to hide negative trials or alter analytic
plans after results are known.
 Reporting standards ensure more transparent
and detailed article so results can be
independently validated.
 Reporting standards and pre-registration of
plans for meta-analyses and systematic reviews.
 Making data available for independent
reanalysis.

7. Our modest contribution to
reform
Roseman, M, Milette, K, Bero, LA, Coyne, JC,
Lexchin, J., Turner, EH, & Thombs, BD. (2011).
Reporting of conflicts of interest in meta-
analyses of trials of pharmacological
treatments. JAMA, 2011;305(10):1008-17.
Roseman, M, Turner, EH, Lexchin, J., Coyne,
JC, Bero, LA, & Thombs, BD. (2012). Reporting
of conflicts of interest from drug trials in
Cochrane reviews: cross sectional study. BMJ,
2012; 345.

9. We documented with association of source of
funding with claimed size of effects.
We proposed considering conflict of interest as
a moderator, source of heterogeneity, and
another risk of bias in Cochrane risk of bias
assessment.
The Cochrane Collaboration agreed.

10. Cochrane Risk of Bias
 Sequence generation.
 Allocation concealment.
 Blinding of participants, investigators, outcome
assessors.
 Incomplete outcome data.
 Selective outcome reporting.
 Other threats to validity.

11. In 1979 he wrote, "It is surely a great criticism of our
profession that we have not organised a critical
summary, by specialty or subspecialty, adapted
periodically, of all relevant randomized controlled trials”
Prof Archibald Cochrane,
CBE (1909 - 1988)
The Cochrane Collaboration is
named in honor of Archie
Cochrane, a British researcher.

12. Clinical psychology and
behavioral medicine
CONSORT adopted later and less consistently
than Biomedicine.*
Preregistration of trials is now encouraged, but
enforcement is lax.
Conflict of interest less acknowledged, although
investigator allegiance pervasive and a risk of
bias.
*APS (Association of Psychological Science) journals not yet
compliant.

14. Trickle down of reform
 Changes forced upon Pharma slowly and
inconsistently reach clinical psychology.
 Larger data sets allow exploration of issues
such as choice of control groups and
investigator allegiance.
 Behavioral Medicine adopts changes
occurring in clinical psychology later and
inconsistently.

15. Clinical psychology and
behavioral medicine
RCT literature dominated by methodologically flawed,
underpowered studies obtaining significant results at
statistically improbable rate.
Weak control groups, unusual to have active control group.
Flexible rules of design and selective reporting of outcomes
chosens after results are known.
Strong investigator allegiance effects.

16.  Evidence-Based Medicine developed to weed
out ineffective treatments.
 Evidence-Based Behavioral Medicine
developed to demonstrate that treatments
worked and should be disseminated and
reimbursed.

17. The Two Tribes of EBBM:
Dodoists and Skeptical Dogs

18. Beware of Professional
Organization Involvement as a Risk
of Bias

19. Guidelines for Guidelines
 Shaneyfelt T. In guidelines we cannot trust. Arch Intern
Med;172:1633-1634.
 Institute of Medicine Committee on Standards for Developing
Trustworthy Clinical Practice Guidelines (2011). Clinical practice
guidelines we can trust. Washington, DC: National Academies
Press.
 Loblaw, D. A., Prestrud, A. A., Somerfield, M. R., et al. (2012).
American Society of Clinical Oncology clinical practice guidelines:
Formal systematic review–based consensus methodology. J Clin
Oncol, 30(25), 3136-3140.

20. AHA Advisory on Screening for
Depression Not Guideline-Congruent!
 Ziegelstein RC, Brett D, Thombs BD, Coyne JC, de
Jonge P. Routine Screening for Depression in Patients
with Coronary Heart Disease: Never Mind. Journal of the
American Academy of Cardiology. 2009;54(10):886-90.
 Thombs, B. D., Jewett, L. R., Knafo, R., Coyne, J. C., &
Ziegelstein, R. C. (2009). Learning from history: a
commentary on the American Heart Association Science
Advisory on depression screening. American heart
journal, 158(4), 503.

21. Beware “Official” meta-analyses
(Critiqued here)
 Coyne JC, Thombs BD, Hagedoorn M. Ain’t Necessarily
So: Review and Critique of Recent Meta-Analyses of
Behavioral Medicine Interventions in Health Psychology.
Health Psychology. 2010;29(2):107-16.
 Coyne, J. C. (2012). Re: Meta-analysis of Efficacy of
Interventions Elevated Depressive Symptoms in Adults
Diagnosed With Cancer. Journal of the National Cancer
Institute, djs408.

22. SBM Initiative
Meta-analyses generated by professional
organizations should receive special
critical scrutiny because of tendency to
gloss over limits of literature in order to
promote the services of their membership.

23. “We are grateful to the Society of Behavioral
Medicine (SBM) for selecting the authorship
group. This article is one of three meta-
analyses that have been undertaken under the
aegis of the SBM Evidence-Based Behavioral
Medicine Committee; the other two meta-
analyses examine the effects of psychosocial
interventions on depression and fatigue among
patients with cancer.”

24. What to watch for in meta analyses
commissioned by professional
organizations
 Precommitment to conclusion that interventions work
and are ready for dissemination and reimbursement.
 Ignoring of preponderance of methodologically flawed,
unpowered sources of bias.
 Broad inclusion of diverse interventions into one
category.
 Ignoring statistical heterogeneity.
 Weak assessment of risk of bias (Cochrane criteria are
seldom used).

25. How This Symposium Came About

26. EHPS 2013: Improving the Credibility of
Health Psychology Intervention Research:
Problems and Solutions
Ozakinci, G., Whitehead, R., Sobota, A., Coyne, JC.
Inaccurate Abstracts in Health Psychology: The Problem
and an Easily Implementable Solution.
Coyne, JC. Too good to be true: Health psychology
depends too much on positive underpowered studies.

27. Picking up collaborators on
the Internet

28. Nick Brown

29. Moritz Heene

30. Daniel Lakens