This document discusses statistical issues related to using patient-reported outcome (PRO) measures in clinical trials. It notes that PROs are often measured on an ordinal scale with skewed distributions and are subject to floor and ceiling effects. Standard analysis methods may not be appropriate. It also discusses challenges like regression to the mean, baseline adjustment, and evaluating change over time without a control group. While a multi-scale PRO like a health-related quality of life instrument can be used as a primary endpoint, its multidimensional nature raises issues around validation and methodology compared to single measures. Regulatory agencies provide guidance on properly using PROs, including multi-scale ones, as primary endpoints.

2. Statistical issues in patient reported
outcome measures
Jonas Ranstam PhD
RC Syd and Lund University, Lund, Sweden,
Email: jonas.ranstam@med.lu.se

3. Science and uncertainty
“If you thought that science was certain - well, that is just an
error on your part.”
Richard P. Feynman

4. Uncertainty in clinical research
Generalization from sample to population

5. Inference (naive)
Real outcome Concluded
outcome
Observed
outcome

6. Inference (scientific)
Real outcome Uncertainty
Concluded
acknowledged
outcome
Observed
outcome
Biological
variation Co-
morbidity
Measurement
errors Effect
modification
Selection Case-
mix
Mis-
classification Interaction

7. The statistician's task
To eliminate as much uncertainty as possible (by design) and
to quantify (in the analysis) what is left.

8. Statistical characteristics of PROs
1. Measured on an ordinal scale
2. Discrete distribution
3. Truncated distribution
4. Skewed distribution
5. Floor and/or ceiling effects
Are standard analysis methods appropriate?

10. Lillgraven S, Kristiansen IS, Kvien TK. Comparison of utility measures and their relationship
with other health status measures in 1041 patients with rheumatoid arthritis. Ann Rheum Dis
2010;69:1762-1767.

12. The EQ-5D index
Simulation studies at RC Syd show that
Type-1 error rate (risk of false positive findings)
EQ-5D is best analyzed using methods that assume a
Gaussian distribution, at least if n is large, between 20-50.
Non-parametric alternatives perform poorly with any n value.
Type-2 error rate (risk of false negative findings)
No single method can be recommended. All investigated
methods perform poorly for any distributional shape.

13. Complications
Longitudinal analyses
(of change, gain, delta-value, etc.)

14. Baseline versus change
Simulated data (n = 1000)
pre - post
correlation (pre, post) = 0
delta = post - pre
correlation (pre, post - pre) = -0.7

15. Baseline versus change
50% of the “change” can be explained by baseline.
When comparing “change” in different groups, always adjust
for imbalance at baseline (e.g. using ANCOVA).

16. RTM - Regression to the mean
If the first measurement of a variable is extreme, the second
measurement will tend to be closer to the average.
Note, this is a purely statistical phenomenon.
Galton F. Regression towards mediocrity in hereditary stature. J Anth Inst Gr
Br Ire.1886;15:246–263.

17. Hypothetical example: SF-36 PF
Baseline: mean = 80, SD = 17
Follow up: mean = 80, SD = 17
p ≈ 1.0

18. Hypothetical example: SF-36 PF
Baseline: mean = 48.7, SD = 8.6
Follow up: mean = 59.2, SD = 16.7
p < 0.001

19. RTM - Regression to the mean
The phenomenon explains the placebo effect in clinical trials
and apparent treatment effects found in some studies on
homeopathic drugs, bible reading, etc.

20. RTM - Easy to quantify
(for Normally distributed endpoints)
Barnett AG, van der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. Int J Epidemiol 2005;34:215–220

21. Hypothetical example of RTM in SF-36 PF
Mean = 80, SD = 17, cut off = 60
r RTM
0.0 28.4
0.1 25.5
0.2 22.7
0.3 19.9
0.4 17.0
0.5 14.2
0.6 11.3
0.7 8.5
0.8 5.7
0.9 2.8
1.0 0

22. Regression-to-the-mean
Evaluation of a single group’s development over time
should be avoided, or at least include a comparison with
the expected RTM effect.

23. Practical consequences
When evaluating change, use a control group.
Adjust for baseline imbalance.
The validity of this procedure with multi-modal data
(e.g. the EQ-5D index) is unknown.

24. Thank you for your attention!

25. Can a multi-scale PRO be used as a
primary endpoint in a randomized trial?

26. Can a multi-scale PRO be used as a
primary endpoint in a randomized trial?

27. Can a PRO be used as a primary
endpoint in a randomized trial?
PRO as primary endpoint
“A PRO measurement can be the clinical trial’s primary
endpoint measure, a co-primary endpoint measure ... or a
secondary endpoint measure whose analysis is considered
according to a hierarchical sequence.“
FDA. Patient-Reported Outcome Measures: Use in Medical Product Development
to Support Labeling Claims. Guidance for Industry.

28. Can a multi-scale PRO be used as a
primary endpoint in a randomized trial?
PRO and HRQL
“The term PRO is proposed as an umbrella term to cover both
single dimension and multi-dimension measures of
symptoms, health-related quality of life (HRQL), health status,
adherence to treatment, satisfaction with treatment, etc.”
“In the context of drug approval, HRQL is considered to
represent a specific type/subset of PROs, distinguished by its
multi-dimensionality.”
EMEA. Reflection paper on the regulatory guidance for the use of health-related
quality of life (HRQL) measures in the evaluation of medicinal products.

29. Can a multi-scale PRO be used as a
primary endpoint in a randomized trial?
HRQL as primary endpoint
“In general, the methodology for assessing the effect on
HRQL is similar to the methodology used in any efficacy trial,
except for issues related to the nature of the instruments,
which are generally composed of multi-items, and multi-
domains.
Briefly, it is recommended that HRQL instrument be
previously validated for the condition studied...”
EMEA. Reflection paper on the regulatory guidance for the use of health-related
quality of life (HRQL) measures in the evaluation of medicinal products.

30. FDA. Patient-Reported Outcome Measures: Use in Medical Product Development to
Support Labeling Claims. Guidance for Industry.