Associate editor for statistics, Osteoarthritis and Cartilage Clinical Epidemiology Unit, Lund University

1. Aleksandra Turkiewicz, PhD, CStat
Associate editor for statistics, Osteoarthritis and Cartilage
Clinical Epidemiology Unit, Lund University
Statistical Review of Basic Science
Manuscripts at Osteoarthritis and
Cartilage – Why and How?

2. Disclosures
• None

3. Deputy editor:
Prof. Jonas Ranstam
Lund University
Associate editor:
Prof. Simon Skene
University of Surrey
Statistical reviewers at OAC

4. Outline
• What does a statistical reviewer want and why?
– Study design and consequences
– Quantifying uncertainty
– Interpretation of findings

5. The logical flow
Study
question
Study design
Statistical
analysis
Results +
interpretation

6. What does a statistical reviewer want to know?
• That there is as little bias as possible
• That the uncertainty is adequately quantified
• How big the difference is

7. The main focus of a statistical reviewer
• Study design!
• The burden of proof lies on the author

8. Study design must ensure validity

9. Validity
• Random sampling from the underlying population
From: http://www.lightmatter.net/gallery/albums.php

10. Validity
• Randomization of treatments
“Randomly” taking an animal
from a cage is not randomization

11. Validity
• Randomization of treatments
Computer generated program where
each animal has equal probability to
receive each treatment

12. Validity
• Cell cultures from cartilage samples: OA and references
• Age and sex are important confounders
• Alternative – matching
OA reference
56 years old, males
76 years old, females

13. Validity
• Blinded conduct of experiment
Surgery (often DMM)
Application of treatments
Handling of samples
https://doi.org/10.1016/j.joca.2015.11.008

14. Validity
• Blinding in the assessment of the outcome – a ‘must have’!
Poster 477

15. Validity
• Exclusions – disclose all

16. O’Collins et al, 2006
In vitro and in vivo - 1026
Tested in vivo - 603
Effective in vivo - 374
Tested in clinical trial - 97
Effective in clinical trial - 1
1026 interventions in experimental stroke
Risks of bias in animal research – and what to about them, Malcolm Macleod

17. • Infarct Volume
– 11 publications, 29 experiments, 408 animals
– Improved outcome by 44% (35-53%)
Efficacy
Randomisation Blinded
assessment of
outcome
Blinded
conduct of
experiment
Risk of bias in animal studies
Macleod et al, 2008
Risks of bias in animal research – and what to about them, Malcolm Macleod

18. Dependent and independent
DOI: 10.1186/2191-219X-1-11
Mouse strains
(often inbred)
and from one
source
https://doi.org/10.1186/s40317-015-0069-0
DOI: 10.1002/art.40351

19. Dependent and independent
DOI: 10.1186/2191-219X-1-11
Mouse strains
(often inbred)
and from one
source
https://doi.org/10.1186/s40317-015-0069-0
DOI: 10.1002/art.40351
Nested dependence structure due to biological relationships

20. Dependent and independent
Knee picture: Martel-Pelletier et al. https://www.nature.com/articles/nrdp201672

21. Dependent and independent
• Nested dependence structure due to
biological relationships and study
design needs to be clearly described in
the manuscript
• Impacts generalizability of the findings
• Needs to be taken into account in the
statistical analysis
Basic science researcher
Multilevel regression

22. Quantifying uncertainty
• Statistical methods are used when there is variation that obscures
the view
Sampling error
Measurement error of the measuring device
Technical variation
Biological variation

23. Quantifying uncertainty
Data

24. Quantifying uncertainty
The difference in mean
outcome between the
treatment and control is:
8
8
We can be pretty sure, that
the difference in the
underlying population is not 8

25. Toss 6 coins
random.org/coins

26. Toss 6 coins
• Did you get exactly 3 ”do its”?

28. Experiment number

29. Quantifying uncertainty
Data
Statistical model and
its assumptions
Study
design
Statistical
analysis

30. Quantifying uncertainty
8
The confidence interval gives us
information about what the difference
in the underlying population may be.
The difference in mean
outcome between the
treatment and control is:
8 (95%CI 2 to 14)

31. Quantifying uncertainty
14
The difference in mean
outcome between the
treatment and control is:
8 (95%CI 2 to 14)

32. Quantifying uncertainty
2
14
The difference in mean
outcome between the
treatment and control is:
8 (95%CI 2 to 14)

33. Quantifying uncertainty
The difference in mean outcome between treatment and control:
8 (95% 2 to 14)
The interpretation of the result needs to take into account the
values included in the 95% (or another relevant) confidence
interval.

34. Quantifying uncertainty

35. For a statistical reviewer…
It doesn’t matter what your p-values is!
It doesn’t matter if the result if ‘statistically significant’

36. Interpretation of findings

37. What is important?
Validity
Assumptions
Uncertainty
Study
question
Study design
Statistical
analysis
Results +
interpretation

38. www.clin-epi.com
Clinical Epidemiology Unit
aleksandra.turkiewicz@med.lu.se
Thank you and questions!
@dr_englund @clinepi_LU