This document discusses measurement error in medical research. It notes that measurement error is common for variables like blood pressure, diet, smoking status, and others. Measurement error can bias results and weaken observed associations. However, measurement error corrections are rarely applied in medical studies, with only a small percentage of studies accounting for it. The document advocates that researchers should be more aware of measurement error and its effects and consider ways to address it in studies.

1. Measurement error in
medical research
Maarten van Smeden, PhD
University Medical Center Utrecht
Julius Center for Health Sciences and Primary Care
The Netherlands
Twitter: @MvanSmeden
Email: M.vanSmeden@umcutrecht.nl
29 Feb 2020
Clinical Studies Application and Education Project Group Meeting (STATISTANBUL 2020)
Turkish Society of Cardiology
Istanbul, Turkey
Sides available at https://www.slideshare.net/MaartenvanSmeden
I have no conflicts of interest to declare

2. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error
“Errors in reading, calculating or recording a
numerical value. The difference between
observed values of a variable recorded
under similar conditions and some fixed true
value.“
The Cambridge Dictionary of Statistics (4th ed), ISBN: 9780521766999

3. Twitter: @MaartenvSmedenIstanbul, February 29 2020 img: https://bit.ly/2T9UnRt

4. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement of systolic blood pressure
Measurement error due to:
• White coat effect1
• Non-adherence to measurement protocol2
• Fallibity of measurement instruments3
• ….
Measurement error varies:
• Number of BP measurements taken4
• Gender4
• Circadian rhythm
• ….
doi: 110.1370/afm.1211; 210.3399/096016407782604965; 310.2147/MDER.S141599; 410.3109/08037051.2014.986952

5. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Example circadian rythm
doi: 10.1111/j.1552-6909.2000.tb02771.x

6. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Imprecision of medical measurements
doi: 10.1136/bmj.m149

7. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error: a long list
• Blood pressure
• Dietary intake
• Smoking status
• Air pollution
• BMI
• Physical activity
• Vaccination status
• Social class
• Carotid intima media thickness
• Thyroid hormone levels
• Glucose levels
• Cholesterol levels
• Income
• Family history
• Mental health history
• Education level
• “Intelligence”
• Sedentary hours
• Vitamin use
• Immigration status
• Age at first intercourse
• ICD coding
• Symptoms
• Medication use
• Visceral adipose tissue
• ….

8. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error mentioned
Journals of epidemiology
Jurek et al. 20061 61% (N = 35)
Brakenhoff et al. 20162 56% (N = 198)
Shaw et al. 20193 80% (N = 65)
doi: 110.1007/s10654-006-9083-0; 210.1016/j.jclinepi.2018.02.02; 310.1016/j.annepidem.2018.09.001

9. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error mentioned
Journals of general medicine
Brakenhoff et al. 20162: 25% (N = 57)
doi: 210.1016/j.jclinepi.2018.02.02

10. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error “corrections” applied
Journals of epidemiology
Jurek et al. 20061: 2% (N = 1)
Brakenhoff et al. 20162: 4% (N = 13)
Shaw et al. 20193: 6% (N = 5)
doi: 110.1007/s10654-006-9083-0; 210.1016/j.jclinepi.2018.02.02; 310.1016/j.annepidem.2018.09.001

11. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error “corrections” applied
Journals of general medicine
Brakenhoff et al. 20162: 0% (N = 0)
doi: 210.1016/j.jclinepi.2018.02.02

12. Twitter: @MaartenvSmedenIstanbul, February 29 2020

13. Twitter: @MaartenvSmedenIstanbul, February 29 2020
• Many studies categorized the continuous variables: dichotomania
– Common myth: categorization will reduce impact of
measurement error
• Many studies made incomplete/incorrect claims
– Many stated that estimates could only be attenuated
(‘underestimated’) due to measurement error
– Some claimed no bias due to measurement error in
associations

14. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Excerpt from a trial protocol
excerpt from: 10.1186/s13063-018-2954-3

15. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Types of measurement error
Measurement are
consistently wrong in a
particular direction
Classical (Random)
measurement error
Differential
measurement error
Systematic
measurement error
Measurements fluctuate
around their true value
Measurements are
consistently wrong in a
particular direction,
varying per group
Courtesy: Linda Nab

16. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Classical measurement error

17. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Classical measurement error

18. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Tripple whammy of measurement error
• Bias
• Increased imprecision
• Masked functional relations

19. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Tripple whammy of measurement error
• Bias
• Increased imprecision
• Masked functional relations
Always weaker effects?

20. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Example: classical measurement error
doi: 10.1371/journal.pone.0192298

21. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Example: classical measurement error
Second Manifestations of ARTerial disease (SMART) cohort
doi: 10.1371/journal.pone.0192298
Effect of
interest
Confounder
with error
Outcome

22. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Example: classical measurement error
doi: 10.1371/journal.pone.0192298
% bias in hazard ratio for SBP (multivariable Cox regression model)

23. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Example: classical measurement error
Second Manifestations of ARTerial disease (SMART) cohort
doi: 10.1371/journal.pone.0192298
Effect of
interest
Confounder
with error
Outcome

24. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Example: classical measurement error
doi: 10.1371/journal.pone.0192298
% bias in hazard ratio for SBP (multivariable Cox regression model)

25. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Randomized controlled trials
doi: 10.1002/sim.8359

26. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Randomized controlled trials
Classical (Random)
measurement error
Systematic
measurement error
Differential
measurement error
doi: 10.1002/sim.8359

27. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Randomized controlled trials
Classical (Random)
measurement error
Systematic
measurement error
Differential
measurement error
• Unbiased treatment effect estimator
• Increased Type-II error
• Nominal Type-I error
• Possibly biased TE estimator
• Type-II error affected
• Type-I generally nominal
• Possibly biased TE estimator
• Type-II error affected
• Type-I not nominal
doi: 10.1002/sim.8359

28. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Tripple whammy of measurement error
• Bias
• Increased imprecision
• Masked functional relations
Usually the target for measurement error “corrections”

29. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error corrections
External validation set
Study sample
!∗
External validation set
Standard
measurements
Standard
measurements
+
Validated
measurements

30. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error corrections
Internal validation set
Study sample
!∗Internal validation set
Standard
measurements
Standard
measurements
+
Validated
measurements

31. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error corrections
Replicates study
Study sample
!∗
Standard
measurements
replicated

32. Twitter: @MaartenvSmedenIstanbul, February 29 2020
A few measurement error models
Measurement of the variables with error (X) with/without truth (T)
• Latent variable models
– replicate measures of X to approximate T
• Regression calibration
– X is replaced by a prediction of T
• SIMEX:
– Error is simulated + added to X, extrapolated to estimate T effect of X
• MIME:
– T is multiple imputed for individuals for whom T wasn't observed
!∗

33. Twitter: @MaartenvSmedenIstanbul, February 29 2020
A few measurement error models
Measurement of the variables with error (X) with/without truth (T)
• Latent variable models
– E.g. R-package lavaan
• Regression calibration
– E.g. R-package mecor (www.github.com/LindaNab/mecor)
• SIMEX:
– E.g. R-package simex
• MIME:
– E.g. R-package mice
!∗

34. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Without replicates or validation data
https://lindanab.shinyapps.io/SensitivityAnalysis/
Preprint: https://arxiv.org/abs/1912.05800

35. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error models are not new
doi: 10.2307/1422689

36. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement error models are not new
doi: 10.2307/2530508

37. Twitter: @MaartenvSmedenIstanbul, February 29 2020 img: https://support.apple.com/en-us/HT208931

38. Twitter: @MaartenvSmedenIstanbul, February 29 2020
419,297 participants
doi: 10.1056/NEJMoa1901183

39. Twitter: @MaartenvSmedenIstanbul, February 29 2020
Measurement: are labels the new oil?
https://twitter.com/DrHughHarvey/status/1230218991026819077

40. Twitter: @MaartenvSmedenIstanbul, February 29 2020 doi: 10.1093/ije/dyz251

41. Twitter: @MaartenvSmedenIstanbul, February 29 2020