Iain Chalmers argues that a significant amount of medical research is wasted due to issues such as: 1) Low priority questions being addressed instead of what matters most to clinicians and patients. 2) Many studies being poorly designed and not learning from existing evidence. 3) Important results from studies often not being published or reported, leading to bias. He advocates for greater public involvement in setting research agendas and priorities to better align research with patient and clinician needs and reduce waste estimated at over $85 billion per year. The James Lind Initiative seeks to address research waste and improve focus on shared uncertainties.

1. Avoiding waste in research:
the role of public involvement
Iain Chalmers
Coordinator, James Lind Initiative
‘Putting people first in research’
INVOLVE Conference
Nottingham, 13 November 2012

2. The skeletons in academic
medicine’s cupboards

3. Questions relevant
Appropriate design Accessible Unbiased and
to clinicians &
and methods? full publication? usable report?
patients?
Low priority questions Over 50% studies Over 50% of studies Over 30% of trial
addressed designed without never published in full interventions not
reference to sufficiently described
Important outcomes systematic reviews of Biased under-
not assessed existing evidence reporting of studies Over 50% of planned
with disappointing study outcomes not
Clinicians and Over 50% of studies results reported
patients not involved fail to take adequate
in setting research steps to reduce Most new research
agendas biases, e.g. not interpreted in the
unconcealed context of systematic
treatment allocation assessment of other
relevant evidence
50 50 50
% % %
85% Research waste = over $85 Billion / year

4. Mismatch of patients’ and researchers’
priorities for osteoarthritis of the knee
Tallon, Chard and Dieppe. Lancet, 2000.

5. Priority treatment outcome from a
survey of patients with rheumatoid
arthritis was not pain
It was fatigue

7. For every ongoing trial being conducted
within the NHS, the UK Clinical Trials
Gateway should aim to provide access to:
• a lay summary
• the patient information sheet
• the WHO 20-item dataset
• the protocol, with links to the systematic
review(s) showing why the trial is needed
• the trial website (if one exists)

8. Reliable, user-friendly
information about specific
ongoing clinical trials is
still NOT generally available

10. 2003-
The James Lind Initiative
Funded by the National Institute of Health
Research and the Medical Research Council
“to promote acknowledgement
of uncertainties about the
effects of treatments, and
research to address them.”

11. Programme of work of
The James Lind Initiative
1. Identifying and publishing uncertainties about the
effects of treatments: UK Database of
Uncertainties about the Effects of Treatments
2. Identifying patients’ and clinicians’ shared
priorities for research about the effects of
treatments: James Lind Alliance
3. Explaining and illustrating the development of fair
tests of treatments in health care: James Lind
Library and Testing Treatments

12. The UK Database of Uncertainties
about the Effects of Treatments
Established to publish uncertainties about
the effects of treatments which cannot
currently be answered by referring to
relevant and reliable, up-to-date
systematic reviews of existing
research evidence

13. UK DUETs draws on three main sources
•Patients', carers' and clinicians' unanswered
questions about the effects of treatments
•Research recommendations in reports of
systematic reviews and clinical guidelines
•Ongoing research, both systematic reviews in
preparation and new 'primary' studies

15. Programme of work of
The James Lind Initiative
1. Identifying and publishing uncertainties about the
effects of treatments: UK Database of
Uncertainties about the Effects of Treatments
2. Identifying patients’ and clinicians’ shared
priorities for research about the effects of
treatments: James Lind Alliance
3. Explaining and illustrating the development of fair
tests of treatments in health care: James Lind
Library and Testing Treatments

16. To increase the focus of the therapeutic
research agenda on questions and
priorities shared by patients and clinicians.
To promote Priority Setting Partnerships
involving patients and clinicians to
identify and promote their shared
priorities for therapeutic research.
To increase general awareness and
understanding of the need to refocus the
therapeutic research agenda.

17. Lester Firkins
Chair, JLA Strategy
& Development Group
Sally Crowe Katherine Cowan
Chair, JLA Monitoring & Editor, JLA Guidebook
Implementation Group
Patricia Atkinson
Administrator,
JLA Secretariat

18. Involving patients, carers and
clinicians in research priority setting
The JLA’s principles
•Inclusive
•Balance of perspectives
•Accessible to all
•Supportive
•Recognising a range of capacities and skills
•Transparent and democratic
•Data sharing
•Agreed protocol
•Declaration of interests
•Neutral facilitation
•Communication and feedback

19. JLA Priority Setting Partnerships
Completed Current
•Asthma •Acne
•Urinary incontinence •Childhood disability
•Vitiligo •Dementia
•Prostate cancer •Dialysis
•Schizophrenia •Head and neck cancer
•Type 1 diabetes •Inflammatory bowel disease
•ENT aspects of balance •Lyme disease
•Life after stroke •Multiple sclerosis
•Eczema •Pressure ulcers
•Tinnitus •Pre-term birth
•Cleft lip and palate •Sight loss and vision

20. Research priority themes [across asthma,
incontinence, vitiligo, eczema, stroke, prostate cancer,
schizophrenia, aspects of balance, and type 1 diabetes]
• Assessment of long-term effects (wanted and
unwanted) of treatments
• Assessment of safety and adverse effects of
treatments
• Assessment of complementary and non-prescribed
treatments
• Assessment of strategies to improve early diagnosis
and treatments, and harmonisation of practice
• Assessment of the effectiveness and safety of self-
care

21. Questions relevant
Appropriate design Accessible Unbiased and
to clinicians &
and methods? full publication? usable report?
patients?
Low priority questions Over 50% studies Over 50% of studies Over 30% of trial
addressed designed without never published in full interventions not
reference to sufficiently described
Important outcomes systematic reviews of Biased under-
not assessed existing evidence reporting of studies Over 50% of planned
with disappointing study outcomes not
Clinicians and Over 50% of studies results reported
patients not involved fail to take adequate
in setting research steps to reduce Most new research
agendas biases, e.g. not interpreted in the
unconcealed context of systematic
treatment allocation assessment of other
relevant evidence
50 50 50
% % %
85% Research waste = over $85 Billion / year

22. Publication (2007) after registration (1999)
Country
Size
Phase
Funder
Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM (2009). Trial publication after
registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 6(9): e1000144.

23. “Studies that report positive or significant results
are more likely to be published and outcomes
that are statistically significant have higher odds
of being fully reported.”
PLoS ONE, August 2008;3:e3081

25. Alessandro Liberati

26. Because research results have not
been made public…
…patients have suffered and died
unnecessarily and resources for health
care and health research have been
wasted.

27. TGN1412
TGN 1412

34. What is the position of the
Academy of Medical Sciences?

35. 2006 letter to Prof John Bell,
President, Academy of Medical Sciences

37. What should be done?
The public needs to be made
aware of how the resources they
provide for research are being
wasted.
The public needs to hold the
research community to account,
and be critically involved in
research, from agenda setting to
dissemination of results.

38. Programme of work of
The James Lind Initiative
1. Identifying and publishing uncertainties about the
effects of treatments: UK Database of
Uncertainties about the Effects of Treatments
2. Identifying patients’ and clinicians’ shared
priorities for research about the effects of
treatments: James Lind Alliance
3. Explaining and illustrating the development of fair
tests of treatments in health care: James Lind
Library and Testing Treatments

39. www.jameslindlibrary.org

41. Promote research on the effects of treatments…
…but only if it meets scientific and ethical principles

42. www.testingtreatments.org

43. “Bad Science introduces the basic scientific principles to
help everyone become a more effective bullshit detector.”