How to improve  the chance of getting your manuscript accepted for publication Jonas Ranstam PhD
 
Anecdotal evidence (Case reports) ‏ Evidence based medicine (The Cochrane collaboration 1993) ‏ Cohort study of smoking  a...
EU directive (2001) ‏ ICH GCP (1996) ‏ CONSORT (1996) ‏ WHO CIOMS (1993) ‏ ICMJE Uniform Requirements (1978) ‏ Helsinki de...
Plan 1. Methodological background 2. General guidelines 3. Special recommendations a) case reports b) mechanical experimen...
1. Methodological background
What is statistics used for? 1. Describing data ‏ (statistics in the plural)‏ 2. Interpreting uncertain data (statistics i...
Two kinds of uncertainty 1. Uncertainty of measurement 2. Uncertainty of sampling
1. Uncertainty of measurement The precision of the used measurement instrument. The precision of the Finapres non-invasive...
2. Uncertainty of sampling Individual effects vary between subjects. Different samples of subjects yield different observe...
Example Assume that the cumulative 10-year revision rate of the Oxford knee prosthesis is 8% and that two groups of 100 pa...
375 randomly ordered patients of which 30 (8%)  will be revised within 10 years
 
6% revised 12% revised Sampling uncertainty
6% revised 12% revised H 0 : The two samples represent the same population H 1 : The two samples represent different popul...
P-value The probability that an observed effect only reflects sampling uncertainty. 12/100 vs. 6/100, Fisher's exact test ...
P-values are often misunderstood They cannot -  describe clinical relevance (they depend on sample   size) ‏ - show that a...
Confidence interval A range of values, which with the specified confidence level describes how likely it is that the estim...
Confidence interval A range of values, which with the specified confidence level describes how likely it is that the estim...
Important assumptions Many statistical methods like the Student's t-test and ANOVA are based on the assumption of Gaussian...
Important assumptions Many statistical methods like the Student's t-test and ANOVA are based on the assumption of Gaussian...
Important assumptions Most conventional methods (both parametric and non-parametric) require independent observations.
Important assumptions Most conventional methods (both parametric and non-parametric) require independent observations.  - ...
Copyright ©1995 BMJ Publishing Group Ltd. Bland, J M. et al. BMJ 1995;310:446 pH against PaCO2 for eight subjects,  with p...
How Many Patients? How Many Limbs? Analysis of Patients or Limbs in the Orthopaedic Literature: A Systematic Review  Bryan...
Important assumptions Most conventional methods (both parametric and  non-parametric) require independent observations.  I...
Multiplicity In contrast to many other forms of precision, statistical precision depends on the number of performed measur...
Multiplicity Each significance test at a 5% significance level has 5% risk of a false positive test. Repeated testing incr...
Example 1 (Subgroups, two tests) ‏
Example 2 (Repeated testing,five tests) ‏
Example 3 (Liver function, 10 tests) ‏
Example 4 (Scores, 135 tests) ‏
Multiplicity Common in exploratory analyses  Unacceptable in confirmatory analyses
2. General guidelines
Statistical Methods “ Describe statistical methods with enough detail to enable a knowledgeable reader with access to the ...
Statistical Methods “ Describe statistical methods with enough detail to enable a knowledgeable reader with access to the ...
Results “ When possible, quantify findings and present them with appropriate indicators of measurement error or uncertaint...
Results “ When possible, quantify findings and present them with appropriate indicators of measurement error or uncertaint...
Results “ Avoid relying solely on statistical hypothesis testing, such as the use of P values, which fails to convey impor...
Results “ Avoid relying solely on statistical hypothesis testing, such as the use of P values, which fails to convey impor...
Clinically Statistically significant significant yes  no yes  a  b   no  c  d There was, or was no, (statistically signifi...
Example Two side effects with a new osteoporosis treatment: - A statistically significant reduction in body hair growth ra...
Confidence intervals are better than p-values In contrast to p-values they do -  relate to clinical significance   - show ...
0 Effect Clinically significant effects Statistically and clinically significant effect Statistically, but not necessarily...
When there is a difference in data Do not write that there is not a difference!
There were indeed differences, they are 0.45 and 0.57
There were indeed differences, they are 0.45 and 0.57 Better alternative: “ The observed differences in extraction torques...
Avoid non-technical use of technical terms and use clear expressions - significant clinically or statistically? - no diffe...
3. Special recommendations
a) case reports
Case reports can be used for - Generation of new hypotheses - Showing inconsistencies in established “facts”
Case reports may need  statistics (in the plural sense)‏ - Summary description of characteristics  - Description of change...
Case reports cannot be used for  - Generalizing findings like risk or treatment effect (This requires statistics in the si...
b) mechanical experiments
Mechanical experiments What do p-values and confidence intervals relate to?  -  Measurement uncertainty (Perhaps) ‏ -  Sam...
c) in vitro/cadaver experiments
In vitro/cadaver experiments What do p-values and confidence intervals relate to?  - Measurement uncertainty (Perhaps) ‏ -...
Example In a study with 60 observations 20 specimens had been taken from each of 3 subjects.  The specimens were distribut...
d) cross-sectional studies
Remember - Sampling frame - Target population Super (for scientific questions) ‏ Finite (requires corrections) ‏ - Non-res...
e) epidemiological studies
Epidemiological studies - Exploratory, hypothesis generating,    multiplicity issues considered less important than validi...
Results Uniform Requirements: “Where scientifically appropriate, analyses of the data by variables such as age and sex sho...
Results Uniform Requirements: “Where scientifically appropriate, analyses of the data by variables such as age and sex sho...
Results Automatic stepwise regression (forward or backward) is  not  an adequate method for confounding adjustment.
f) randomized trials
Clinical trials “ The ICMJE member journals will require, as a condition of consideration for publication in their journal...
Clinical trials “ When reporting experiments on human subjects, authors should indicate whether the procedures followed we...
WORLD MEDICAL ASSOCIATION DECLARATION OF HELSINKI Ethical Principles for Medical Research Involving Human Subjects 27.  .....
Purpose of a randomized trial To test a hypothesis with control of random and systematic errors. - No bias (randomization ...
Randomization Mathematical algorithm Stratified Concealment of outcome Reproducible
Study populations Intention-to-treat Analyze all randomized subjects (ITT) principle  according to planned treatment  regi...
FAS vs. PP-set FAS + no selection bias - misclassification problem (effect dilution) ‏ PP-set + no contamination problem -...
Endpoints Primary The variable capable of providing the most clinically relevant evidence directly related to the primary ...
Statistical analyses Confirmatory The result concerns a primary endpoint and the p-value or confidence interval  accounts ...
Reporting “ For reports of randomized controlled trials authors should refer to the CONSORT statement.”
 
 
 
Include with the manuscript Study Protocol Statistical Analysis Plan
Clinical trials International regulatory guidelines  ICH Topic E9 - Statistical Principles for Clinical Trials EMEA Points...
4. Summary
The responsibilities of a statistical reviewer “ To make sure that the authors spell out for the reader the limitations im...
Read the manuscript from end to beginning, and look for weaknesses in the links between: 1. Conclusion 2. Discussion (Disc...
Summary 1. Present statistical methods in detail, and the number of observations included in each analysis.  2. Present da...
Thank you for your attention!
Upcoming SlideShare
Loading in...5
×

Copenhagen 23.10.2008

298

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
298
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Copenhagen 23.10.2008

  1. 1. How to improve the chance of getting your manuscript accepted for publication Jonas Ranstam PhD
  2. 3. Anecdotal evidence (Case reports) ‏ Evidence based medicine (The Cochrane collaboration 1993) ‏ Cohort study of smoking and lung cancer (1954) ‏ (Bradford Hill) ‏ Case-control study of smoking and lung cancer (1950) ‏ (Bradford Hill) ‏ Randomised clinical trial of streptomycin and tubercolosis (1948) ‏ (Bradford Hill) ‏
  3. 4. EU directive (2001) ‏ ICH GCP (1996) ‏ CONSORT (1996) ‏ WHO CIOMS (1993) ‏ ICMJE Uniform Requirements (1978) ‏ Helsinki declaration (1964) ‏ Nürnberg convention (1949) ‏ Trial registration (2005)‏ Mandatory disclosure of trial results (2008)‏
  4. 5. Plan 1. Methodological background 2. General guidelines 3. Special recommendations a) case reports b) mechanical experiments c) in vitro/cadaver experiments d) cross-sectional studies e) epidemiological studies f) randomized trials 4. Summary
  5. 6. 1. Methodological background
  6. 7. What is statistics used for? 1. Describing data ‏ (statistics in the plural)‏ 2. Interpreting uncertain data (statistics in the singular)‏
  7. 8. Two kinds of uncertainty 1. Uncertainty of measurement 2. Uncertainty of sampling
  8. 9. 1. Uncertainty of measurement The precision of the used measurement instrument. The precision of the Finapres non-invasive blood pressure monitor is on the average 12.1 mm Hg.
  9. 10. 2. Uncertainty of sampling Individual effects vary between subjects. Different samples of subjects yield different observed mean effects.
  10. 11. Example Assume that the cumulative 10-year revision rate of the Oxford knee prosthesis is 8% and that two groups of 100 patients receiving the prosthesis are randomly selected and followed over time. The two groups are likely to get different numbers of patients revised during follow up.
  11. 12. 375 randomly ordered patients of which 30 (8%) will be revised within 10 years
  12. 14. 6% revised 12% revised Sampling uncertainty
  13. 15. 6% revised 12% revised H 0 : The two samples represent the same population H 1 : The two samples represent different populations
  14. 16. P-value The probability that an observed effect only reflects sampling uncertainty. 12/100 vs. 6/100, Fisher's exact test p = 0.22
  15. 17. P-values are often misunderstood They cannot - describe clinical relevance (they depend on sample size) ‏ - show that a difference “does not exist”, because n.s. is absence of evidence, not evidence of absence
  16. 18. Confidence interval A range of values, which with the specified confidence level describes how likely it is that the estimated population parameter is included. 12/100 vs. 6/100, RR = 2.0 (95%Ci: 0.7 - 5.6) ‏ 1 Relative Risk 2 1/2
  17. 19. Confidence interval A range of values, which with the specified confidence level describes how likely it is that the estimated population parameter is included. 12/100 vs. 6/100, RR = 2.0 (95%Ci: 0.7 - 5.6) ‏ 1 Relative Risk 2 1/2 p < 0.05 n.s.
  18. 20. Important assumptions Many statistical methods like the Student's t-test and ANOVA are based on the assumption of Gaussian distribution and homogeneous variance.
  19. 21. Important assumptions Many statistical methods like the Student's t-test and ANOVA are based on the assumption of Gaussian distribution and homogeneous variance. If the assumptions are not met, use alternative (non-parametric) methods, like the Mann-Whitney U-test or Kruskal-Wallis non-parametric anova).
  20. 22. Important assumptions Most conventional methods (both parametric and non-parametric) require independent observations.
  21. 23. Important assumptions Most conventional methods (both parametric and non-parametric) require independent observations. - Patients are independent - Patients' knees, hips, shoulders, feet, etc. are not
  22. 24. Copyright ©1995 BMJ Publishing Group Ltd. Bland, J M. et al. BMJ 1995;310:446 pH against PaCO2 for eight subjects, with parallel lines fitted for each subject Incorrect analysis: r = -0.51, p < 0.001 Correct analysis: r = -0.07, p = 0.7
  23. 25. How Many Patients? How Many Limbs? Analysis of Patients or Limbs in the Orthopaedic Literature: A Systematic Review Bryant et al. JBJS Am. 2006;88:41-45. Our findings suggest that a high proportion (42%) of clinical studies in high-impact-factor orthopaedic journals involve the inappropriate use of multiple observations from single individuals, potentially biasing results. Orthopaedic researchers should attend to this issue when reporting results.
  24. 26. Important assumptions Most conventional methods (both parametric and non-parametric) require independent observations. Include only one observation per patient, or use a statistical method that can handle dependant data, e.g. multilevel or mixed effects models. Always present both number of observations and patients.
  25. 27. Multiplicity In contrast to many other forms of precision, statistical precision depends on the number of performed measurements (significance tests).
  26. 28. Multiplicity Each significance test at a 5% significance level has 5% risk of a false positive test. Repeated testing increases the risk of at least one false positive test. Number of tests Risk of at least one false positive 1 0.05 2 0.10 5 0.23 10 0.40
  27. 29. Example 1 (Subgroups, two tests) ‏
  28. 30. Example 2 (Repeated testing,five tests) ‏
  29. 31. Example 3 (Liver function, 10 tests) ‏
  30. 32. Example 4 (Scores, 135 tests) ‏
  31. 33. Multiplicity Common in exploratory analyses Unacceptable in confirmatory analyses
  32. 34. 2. General guidelines
  33. 35. Statistical Methods “ Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to verify the reported results.”
  34. 36. Statistical Methods “ Describe statistical methods with enough detail to enable a knowledgeable reader with access to the original data to verify the reported results.” Required for analytical methods (statistical models, hypothesis tests, confidence intervals). Descriptions are often unclear, vague or ambiguous. They need to be clear and detailed.
  35. 37. Results “ When possible, quantify findings and present them with appropriate indicators of measurement error or uncertainty (such as confidence intervals).”
  36. 38. Results “ When possible, quantify findings and present them with appropriate indicators of measurement error or uncertainty (such as confidence intervals).” Statistical precision (p-values and confidence inter-vals) are necessary for generalization of results beyond examined patients.
  37. 39. Results “ Avoid relying solely on statistical hypothesis testing, such as the use of P values, which fails to convey important information about effect size.”
  38. 40. Results “ Avoid relying solely on statistical hypothesis testing, such as the use of P values, which fails to convey important information about effect size.” Describe both your observations and how you interpret them (use confidence intervals or p-values).
  39. 41. Clinically Statistically significant significant yes no yes a b no c d There was, or was no, (statistically significant) difference is too simplistic
  40. 42. Example Two side effects with a new osteoporosis treatment: - A statistically significant reduction in body hair growth rate by 5% (p = 0.04) ‏ - A statistically insignificant increase in systolic blood pressure by 25 mmHg (p = 0.06) ‏
  41. 43. Confidence intervals are better than p-values In contrast to p-values they do - relate to clinical significance - show when a difference “does not exist” because they present lower and upper limits of potential clinical effects/differences
  42. 44. 0 Effect Clinically significant effects Statistically and clinically significant effect Statistically, but not necessarily clinically, significant effect Inconclusive Neither statistically nor clinically significant effect Statistically significant reversed effect p < 0.05 p < 0.05 n.s. n.s. p < 0.05 P-values Conclusion from confidence intervals [2 alternatives] [6 alternatives] P-value and confidence interval Statistically but not clinically significant effect p < 0.05
  43. 45. When there is a difference in data Do not write that there is not a difference!
  44. 46. There were indeed differences, they are 0.45 and 0.57
  45. 47. There were indeed differences, they are 0.45 and 0.57 Better alternative: “ The observed differences in extraction torques between the two types of uncoated distal pins can be explained by chance.”
  46. 48. Avoid non-technical use of technical terms and use clear expressions - significant clinically or statistically? - no difference statistically insignificant? - statistical difference statistically significant? - matched selected or just comparable? - correlation relation, regression? - normal Gaussian distribution? - random mathematical algorithm? - etc.
  47. 49. 3. Special recommendations
  48. 50. a) case reports
  49. 51. Case reports can be used for - Generation of new hypotheses - Showing inconsistencies in established “facts”
  50. 52. Case reports may need statistics (in the plural sense)‏ - Summary description of characteristics - Description of change or variation over time
  51. 53. Case reports cannot be used for - Generalizing findings like risk or treatment effect (This requires statistics in the singular sense)
  52. 54. b) mechanical experiments
  53. 55. Mechanical experiments What do p-values and confidence intervals relate to? - Measurement uncertainty (Perhaps) ‏ - Sampling uncertainty (No, there is no information on subject variation. The findings cannot be generalized beyond the device).
  54. 56. c) in vitro/cadaver experiments
  55. 57. In vitro/cadaver experiments What do p-values and confidence intervals relate to? - Measurement uncertainty (Perhaps) ‏ - Sampling uncertainty (Perhaps, if the observations provide information on variation between subjects) ‏
  56. 58. Example In a study with 60 observations 20 specimens had been taken from each of 3 subjects. The specimens were distributed randomly between one control group and one experimental group. What do significance tests of these two groups tell us?
  57. 59. d) cross-sectional studies
  58. 60. Remember - Sampling frame - Target population Super (for scientific questions) ‏ Finite (requires corrections) ‏ - Non-responders
  59. 61. e) epidemiological studies
  60. 62. Epidemiological studies - Exploratory, hypothesis generating, multiplicity issues considered less important than validity issues - External validity (source of subjects) ‏ - Internal validity (confounding) ‏
  61. 63. Results Uniform Requirements: “Where scientifically appropriate, analyses of the data by variables such as age and sex should be included.”
  62. 64. Results Uniform Requirements: “Where scientifically appropriate, analyses of the data by variables such as age and sex should be included.” Observational studies require adjustment for known and suspected confounding factors to produce valid effect estimates. This adjustment is usually performed using statistical modelling (e.g. ANCOVA or regression analysis). The purpose is to increase validity.
  63. 65. Results Automatic stepwise regression (forward or backward) is not an adequate method for confounding adjustment.
  64. 66. f) randomized trials
  65. 67. Clinical trials “ The ICMJE member journals will require, as a condition of consideration for publication in their journals, registration in a public trials registry.” “ The ICMJE recommends that journals publish the trial registration number at the end of the Abstract.”
  66. 68. Clinical trials “ When reporting experiments on human subjects, authors should indicate whether the procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000 (5).”
  67. 69. WORLD MEDICAL ASSOCIATION DECLARATION OF HELSINKI Ethical Principles for Medical Research Involving Human Subjects 27. ...Reports of experimentation not in accordance with the principles laid down in this Declaration should not be accepted for publication.
  68. 70. Purpose of a randomized trial To test a hypothesis with control of random and systematic errors. - No bias (randomization & blinding) ‏ - No multiplicity problems
  69. 71. Randomization Mathematical algorithm Stratified Concealment of outcome Reproducible
  70. 72. Study populations Intention-to-treat Analyze all randomized subjects (ITT) principle according to planned treatment regimen. Full analysis set The set of subjects that is as close (FAS) as possible to the ideal implied by the ITT-principle. Per protocol The set of subjects who complied (PP) set with the protocol sufficiently to ensure that they are likely to exhibit the effects of treatment according to the underlying scientific model.
  71. 73. FAS vs. PP-set FAS + no selection bias - misclassification problem (effect dilution) ‏ PP-set + no contamination problem - possible selection bias (confounding) ‏ When the FAS and PP-set lead to essentially the same conclusions, confidence in the trial is supported.
  72. 74. Endpoints Primary The variable capable of providing the most clinically relevant evidence directly related to the primary objective of the trial Secondary Either measurements supporting the primary endpoint or effects related to secondary objectives
  73. 75. Statistical analyses Confirmatory The result concerns a primary endpoint and the p-value or confidence interval accounts for potential multiplicity. The result can support a claim of superiority, equivalence or non- inferiority. Exploratory All other analyses. The result is either supporting or explanatory, or simply just a new hypothesis.
  74. 76. Reporting “ For reports of randomized controlled trials authors should refer to the CONSORT statement.”
  75. 80. Include with the manuscript Study Protocol Statistical Analysis Plan
  76. 81. Clinical trials International regulatory guidelines ICH Topic E9 - Statistical Principles for Clinical Trials EMEA Points to consider: baseline covariates - missing data - multiplicity issues - etc. and similar documents from the FDA These guidelines can all be found on the internet.
  77. 82. 4. Summary
  78. 83. The responsibilities of a statistical reviewer “ To make sure that the authors spell out for the reader the limitations imposed upon the conclusions by the design of the study, the collection of data, and the analyses performed.” Shor S. The responsibilities of a statistical reviewer. Chest 1972;61:486-487.
  79. 84. Read the manuscript from end to beginning, and look for weaknesses in the links between: 1. Conclusion 2. Discussion (Discussion section) ‏ 3. Results (Results section) ‏ 4. Methods (Material & methods section) ‏ 5. Data (Material & methods section) ‏ 5. Hypothesis (Introduction) ‏ Make sure the chain holds all the way!
  80. 85. Summary 1. Present statistical methods in detail, and the number of observations included in each analysis. 2. Present data, statistical results and your conclusions - data description vs. results interpretation - clinical vs. statistical significance - absence of evidence is not evidence of absence 3. Adjust for confounding factors in observational studies (but do not use stepwise regression) ‏ 4. Comply with the CONSORT checklist in randomized studies
  81. 86. Thank you for your attention!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×