1. Science = generalizable knowledge
Predictive and reliable information
Collected in some subjects and generalized to others
Sampling problems often exists, but not always
2. Qualitative research methods
Are only meaningful without sampling problems
(constant outcome, deterministic events)
or when sampling problems are irrelevant
(one observation is sufficient to reject the hypothesis)
3.
4. Quantitative methods (statistical)
Are used to quantify sampling uncertainty
Sampling uncertainty is caused by variability
Statistics is primarily about variability
11. The Gaussian distribution
Defined by: µ and σ (the latter usually assumed constant)
Abraham de Moivre (1667–1754)
9/24/11
12. Empirical EQ-5D distribution
Xie F Li S-C, Luo N, LO N-N,Yeo S-J,Yang KY, Fong KY, Thumboo J. Comparison
of the EuroQol and Short Form 6D in Singapore Multiethnic Asian Knee
Osteoarthritis Patients Scheduled for Total Knee Replacement. Arthritis &
Rheumatism (Arthritis Care & Research) 2007;57:1043–1049
9/24/11
14. Hospital A Hospital B
87%
13% 70% 30%
!
Mean = 0.58 Mean = 0.58
SD = 0.21 SD = 0.21
9/24/11
15. Studying change in EQ-5D
It has been suggested that pairwise differences
between pre- and postoperative EQ-5D values are
normally distributed and can be meaningfully
interpreted.
9/24/11
16. Studying change in EQ-5D
It has been suggested that pairwise differences
between pre- and postoperative EQ-5D values are
normally distributed and can be meaningfully
interpreted.
It can easily be shown that this is not correct
The sum of two bimodal distribution has a distribution
with three modes, the difference four.
9/24/11
17. Empirical EQ-5D data from knee patients in Trelleborg 2007-2008
Preop EQ-5D Postop EQ-5D
Delta EQ-5D
Delta EQ-5D
9/24/11
22. Additional problem with analyses of change
Change is confounded by association with baseline
X = pre-operative (baseline) value
Y = postoperative (follow up) value
Y-X correlates with X
Solution
When analyzing change, adjust for imbalance at baseline
(This is an almost perfect case-mix adjustment!)
9/24/11
23. 1. Stockholm
2. Kronoberg 2. Gävleborg
3. Östergötland
9/24/11
24. 2. Gävleborg
1. Stockholm
3. Östergötland
2. Kronoberg
9/24/11
26. EQ-5D Problems
Conventional analyses
Mean values not interpretable
Confidence intervals not reliable (Calculated assuming Gaussian
distribution)
P-values not reliable (Student's t-test, ANOVA, etc. requires
Gaussian distribution and homogeneous variance)
9/24/11
27. EQ-5D Problems cont'd
Non-parametric analysis
Median value may not exist.
Confidence intervals not reliable (calculated assuming Gaussian
or binomial distribution).
P-values not reliable (Wilcoxon's MPSR-test requires a
symmetrical distribution, Mann-Whitney U-test requires
distributions with identical shape.
9/24/11
28. EQ-5D Problems cont'd
Adjusting for baseline
How meaningful is the outcome of an ANCOVA with
variables having non-Gaussian, multimodal distributions
(with different number of modes)? What do these
residuals look like?
9/24/11
29. EQ-5D Problems cont'd
Alternative analyses methods?
- Mixture distribution analysis (mixdist library for R)
- Multi-state Markov analysis (msm library for R)
9/24/11
34. “This is about clinical improvement, not science”
9/24/11
35. Swedish law defines clinical improvement work
(CIW) as “not research”
Some CIW projects include experiments on patients
- No ethics approval is required (or can be applied for)
- No informed consent
- No scientific planning or evaluation of the experiments
- No formal publication of studies and results
9/24/11
36. Regression analysis
- Adjusting for baseline
- Models only including statistically significant factors
- Stepwise regression methods
9/24/11
37. What factors should be included in a
linear model (ANCOVA)?
Y = b0 + b1X1 + b2X2 + … + bnXn + e
This is a multiple or multivariable analysis but not multivariate.
Xi is a variable (factor or covariate)
bi is the effect on Y of one unit change in Xi
Assume that Y is blood pressure and X1 an indicator of anti-
hypertensive treatment. bi will then estimate the treatment effect in
terms of blood pressure reduction.
9/24/11
38. Linear models
Answer
It depends on a) the purpose of the study and b) the study design
used.
1. Purpose: (black-box) prediction
Any variable can be included as long as it increases the sensitivity and
specificity of the prediction, and as long as results (bi) are not
interpreted in terms of causal effects.
2. Purpose: effect estimation
The variables needed to produce valid (bi and their s.e.) should be
included.
9/24/11
39. Linear models
1. Common for all designs
Include baseline when analyzing change in a continuous variable.
2. Randomized trial
Include randomization stratification factors (for valid standard errors).
3. Observational study
Include potential confounding factors (for valid regression coefficients).
9/24/11
40. Linear models
How should confounding factors be included?
1. By the investigator's reasoning.
2. By reviewing other publications on the same endpoint.
3. By performing sensitivity analyses.
4. But not by using hypothesis testing or stepwise regression analysis.
9/24/11
41. Parsons et al. A systematic survey of the quality
of research reporting in general orthopaedic
journals. J Bone Joint Surg Br 2011;93-B,1154-9
9/24/11
