Mortality measured at a particular time point (landmark mortality) is often regarded as the gold standard outcome for randomised controlled trials in Intensive Care Medicine. An important limitation of many Intensive Care Medicine trials is that they hypothesize large and potentially implausible reductions in absolute mortality. This is a major problem in trial design for two reasons. Firstly, it makes false negative trial results more likely. Secondly, the less plausible a postulated mortality reduction is the more likely that a statistically significant mortality difference will represent a false positive. This is because a p-value is defined as the probability of finding a result equal to or more extreme than that actually observed, under the assumption that the null hypothesis is true. This means that the greater the pre-trial chance or prior probability that the null hypothesis is correct, the lower the chance that a p-value below a particular significance threshold will represent a true positive. The biggest single problem with the current evidence base is that most hypotheses being tested have low prior probability. We need a new research paradigm to address this problem, particularly in relation to the fundamentals of Intensive Care Medicine. Intensive Care therapy is fundamentally about providing supportive care. Such care includes airway support, oxygen therapy, ventilation therapy, haemodynamic support, fluid therapy, temperature control, and nutritional therapy among others. Setting the goals for these therapies is what intensive care doctors do every day. At present, for most of these treatments, the level of evidence on which we are making our decisions is extremely limited. Moreover, the illusion of physiological gain may be leading us astray; making us believe we know the right thing to do when we really do not. We should be creating systems in our intensive care units that allow us to learn iteratively from every patient so that we can systematically reduce mortality over time by understanding how to optimise supportive care. Bayesian adaptive platform trials using response adaptive randomisation can improve the outcomes of patients with mathematical precision but require us, as doctors, to stop believing we know the answers when we really do not.

1. making trials
better
@dogICUma
paul.young@ccdhb.org.nz

2. power

3. P<0.05

4. assuming that the null
hypothesis is correct...

5. the probability of
obtaining a result equal to
or more extreme than
what was actually
observed

6. The Frequentist Approach
experimental
hypothesis
null
hypothesis
get
data
calculate the probability
of the observed result
under the assumption
that the null hypothesis
is correct

7. apply threshold
P value
P≥0.05
 Accept null hypothesis
P<0.05
 reject null hypothesis

9. 10,000

10. 2,000,000,000

11. BASE RATE
NEGLECT

12. Alcohol & Driving

13. 100% 5%FP

14. 95%

15. 1 out of every 1000
drivers is drunk
1 true
positive
from 999 non-
drunk drivers
expect ≈49 false
positives
1/50 are
drunk = 2%

16. 100 trials

17. true
hypotheses

18. true
positives

19. false
positives

20. 67% chance that a
statistically
significant result is a
true positive

21. 17% chance that a
statistically
significant result is a
true positive

23. If the PRIOR
PROBABILITY that a
treatment effect size is
possible ZERO...

24. the POSTERIOR
PROBABILITY must
equal ZERO

25. Most ICU trial
hypotheses are
long shots

26. no
difference
false
positive

28. An epidemic of
marginal P values

29. P VALUES
SUCK

30. PROBABILIT
Y

31. uncertainty

32. idiosyncratic
practice
variation

33. cognitive
biases

34. Innovative
therapies

35. collateral
benefits

36. hedging one’s
bets

37. best future
treatment

38. a system to optimize
treatment and reliably
improve outcomes over
time

39. The Bayesian Approach
experimental
hypothesis
null
hypothesis
get
data
probability of observed
data occurring if the
hypothesis is correct
probability of observed
data occurring if the null
hypothesis is correct

40. The Bayesian Approach
prior
probability
that the
hypothesis
is true
multiply
by Bayes
Factor
posterior probability
that the hypothesis is
true

42. Prior
probability
Accumulated
data
Posterior
probability

43. RANDOMISATION
IS GOOD
FOR PATIENTS

44. Innovative
therapies

48. fusion of trial science
and quality
improvement

49. Posterior
probability ≥0.99
that a treatment is
superior
Posterior probability
of ≥0.90 of having a
mortality within 1%
treatment
works
treatments
equivalent

50. A statistically robustresult
never

51. optimising ICU care is
a priority for global public
health

52. save money by not
using treatments which are
ineffective

53. saves lives by
incorporating treatments that save
lives into standard care

54. @DogICUma
paul.young@ccdhb.org.nz