This document provides 10 recommendations for transparent and consistent scientific research and publication. It recommends (1) stating the research question and purpose, (2) describing the data source and sample size, (3) presenting individual observations when possible, (4) reporting numbers, averages, and variation when data is aggregated, (5) describing statistical methods used, (6) addressing assumptions of those methods, (7) distinguishing statistical and practical significance and clarifying hypotheses, (8) presenting confidence intervals rather than just p-values, (9) explaining departures from conventional significance levels, and (10) ensuring conclusions are consistent with the statistical rigor. The document emphasizes using clear and precise language to describe methodology, results, and conclusions.

1. Transparency and consistency
Jonas Ranstam PhD

2. Scientific research
A systematic investigation ... designed to develop or contribute to
generalizable knowledge1.
Generalizable: Having predictive and reliable results.
When sampling errors don't exist or are irrelevant, qualitative
research methods (e.g. case reporting) can be used.
If sampling errors do exist, the unavoidable sampling uncertainty
must be quantified (quantitative research) and presented, usually
in terms of p-values and confidence intervals.
1
The US National Science Foundation

3. Statistics
Medical researchers rely as never before on statistics for
generating and testing hypotheses and for estimating risks
and benefits of old and new therapies.
Journals can facilitate the writing and reading of research
reports by implementing clear guidelines for manuscript
preparation.

4. Milestones in scientific publication
1658 – the first scientific journals
1858 – the IMRAD structure
1957 – the abstract
1978 – the Vancouver convention (ICMJE)
1987 – the structured abstract
1997 – the CONSORT guidelines
2007 – the STROBE guidelines

6. Ten recommendations
1. Purpose
2. Data source
3. Observations
4. Descriptions
5. Methods
6. Assumptions
7. Significance
8. Confidence
9. Multiplicity
10. Claims

7. 1. Purpose
State the research question and the purpose of the study. Is
the ambition to describe an observation, to generate
hypotheses or to test a pre-specified hypothesis?

8. 1. Purpose
State the research question and the purpose of the study. Is
the ambition to describe an observation, to generate
hypotheses or to test a pre-specified hypothesis?
Bad
We have shown that the success rate differs between two
common techniques for autologous chondrocyte implantation.

9. 1. Purpose
State the research question and the purpose of the study. Is
the ambition to describe an observation, to generate
hypotheses or to test a pre-specified hypothesis?
Good
We designed an experiment to test the hypothesis of identical
success rates of two common techniques for autologous
chondrocyte implantation.

10. 2. Data source
Describe the source of subjects, cadavers, animals, tissues,
cell line, etc. and how many of these units have been included
in the study.

11. 2. Data source
Describe the source of subjects, cadavers, animals, tissues,
cell line, etc. and how many of these units have been included
in the study.
Bad
We collected 36 pieces of human cartilage.

12. 2. Data source
Describe the source of subjects, cadavers, animals, tissues,
cell line, etc. and how many of these units have been included
in the study.
Good
Three pieces of cartilage from each of twelve physically active
men between 25 and 75 years of age, previously included as
healthy controls in a clinical trial (ref.), were collected for this
study.

13. 3. Observations
When observations can be presented individually, either
numerically or graphically, this should be preferred. With fewer
than 4 observations it should be the rule.

14. 3. Observations
When observations can be presented individually, either
numerically or graphically, this should be preferred. With fewer
than 4 observations it should be the rule.
Bad

15. 3. Observations
When observations can be presented individually, either
numerically or graphically, this should be preferred. With fewer
than 4 observations it should be the rule.
Good

16. 4. Descriptions
When presenting data in aggregated form, always present the
number of included observations as well as their average and
dispersion. If repeated measurements or replicates are
included, present both the number of independent samples and
the total number of observations.

17. 4. Descriptions
When presenting data in aggregated form, always present the
number of included observations as well as their average and
dispersion. If repeated measurements or replicates are
included, present both the number of independent samples and
the total number of observations.
Bad
The mean change in total knee cartilage volume was 0.62 ml.

18. 4. Descriptions
When presenting data in aggregated form, always present the
number of included observations as well as their average and
dispersion. If repeated measurements or replicates are
included, present both the number of independent samples and
the total number of observations.
Good
The mean change in total knee cartilage volume was 0.62
±1.3 ml (n=24).

19. 5. Methods
Describe all used statistical methods in a statistics section. Use
the original names of the methods. These are not always the
same as the names used in software packages.

20. 5. Methods
Describe all used statistical methods in a statistics section. Use
the original names of the methods. These are not always the
same as the names used in software packages.
Bad
We used the independent groups t-test in the group comparison.

21. 5. Methods
Describe all used statistical methods in a statistics section. Use
the original names of the methods. These are not always the
same as the names used in software packages.
Good
We used Satterthwaite's t-test in the group comparison.

22. 6. Assumptions
The validity of statistical results rely on certain assumptions
being fulfilled. Were they?

23. 6. Assumptions
The validity of statistical results rely on certain assumptions
being fulfilled. Were they?
The man of science has learned to believe in justification, not by
faith, but by verification.
Thomas Huxley, 1866

24. 6. Assumptions
The validity of statistical results rely on certain assumptions
being fulfilled. Were they?
Good
The ANOVA residual was examined using a normal probability
plot, which indicated a Gaussian distribution.
The homogeneity of variance was tested using Levene's test.
The assumption of proportional hazards was investigated using
hypothesis tests of Schoenfeld residuals.

25. 7. Significance
A p-value describes the uncertainty in the generalization (the
outcome of a hypothesis test), and has no relevance for the
observed sample itself.
Distinguish between practical and statistical significance. Clarify
what hypotheses are tested.

26. 7. Significance
A p-value describes the uncertainty in the generalization (the
outcome of a hypothesis test), and has no relevance for the
observed sample itself.
Distinguish between practical and statistical significance. Clarify
what hypotheses are tested.
Bad
There was no difference in mean systolic blood pressure
between treated patients (190 mmHg) and controls (135 mmHg)
(p = 0.06).

27. 7. Significance
A p-value describes the uncertainty in the generalization (the
outcome of a hypothesis test), and has no relevance for the
observed sample itself.
Distinguish between practical and statistical significance. Clarify
what hypotheses are tested.
Good
Treated patients had in this study higher mean systolic blood
pressure than controls, 190 vs. 135 mmHg. The observation,
even if not statistically significant (p = 0.06), raises concern for
future treatment.

28. 8. Confidence
The uncertainty in the generalization of a finding is often better
presented using the two limits of a confidence interval, indicating
plausible values, than one probability of a false positive
conclusion.

29. 8. Confidence
The uncertainty in the generalization of a finding is often better
presented using the two limits of a confidence interval, indicating
plausible values, than one probability of a false positive
conclusion.
Bad
The reproducibility was high (ICC = 0.91; p < 0.0001).

30. 8. Confidence
The uncertainty in the generalization of a finding is often better
presented using the two limits of a confidence interval, indicating
plausible values, than one probability of a false positive
conclusion.
Good
The reproducibility was high (ICC = 0.91; 95%Ci: 0.64 - 0.98).

31. 9. Multiplicity
All departures from the conventional levels of 5% significance
and 95% confidence, like the ones achieved by using one-sided
tests, Bonferroni corrections, and simultaneous confidence
intervals, should be explained and motivated.

32. 9. Multiplicity
All departures from the conventional levels of 5% significance
and 95% confidence, like the ones achieved by using one-sided
tests, Bonferroni corrections, and simultaneous confidence
intervals, should be explained and motivated.
Bad
We have in this randomized trial shown that patients born under
the astrological sign of Gemini benefit aspirin treatment more
than others.

33. 9. Multiplicity
All departures from the conventional levels of 5% significance
and 95% confidence, like the ones achieved by using one-sided
tests, Bonferroni corrections, and simultaneous confidence
intervals, should be explained and motivated.
Good
When multiplicity issues were taken into account, we were
unable to find any interaction between astrological sign and
benefit from aspirin treatment.

34. 10. Claims
The level of statistical rigor (precision and addressed uncertainty
issues) should be consistent with the author's purpose and
conclusions.

35. What is all this fuss about confidence
intervals and clinical significance?
Questions that can be answered using p-values
- Can I be sure that there is an effect?
Questions that can be answered using confidence intervals
- Can I be sure that there is an effect?
- Can I be sure that there isn't an effect?
- What effect is there?

36. P-values
Statistical significance
p < 0.05 or n.s.
Confidence intervals
Statistical and clinical significance
Effect
0
Clinically significant effect

37. Statements that should be avoided
- “Statistical difference”
- “Significant difference”
- “There was no difference”
- “ns” and “p > 0.05”
- “p < 0.03”

38. Thank you for your attention