This document proposes using direct clinical tolerance limits rather than the Bland-Altman method to assess agreement between two measurement methods. The direct method avoids assumptions of normality and is not affected by outliers. It determines the percentage of differences within pre-specified clinical tolerance limits. In an example comparing glucometer and venous glucose readings, 90% of differences were within tolerances of -2 to +5 mg/dL, demonstrating sufficient agreement. The direct method allows for asymmetric and proportional tolerance limits not possible with Bland-Altman.

1. Direct of Use of Clinical
Tolerance Limits for Assessing
Agreement: A Robust
Nonparametric Approach
Abhaya Indrayan
PhD(OhioState),FAMS,FRSS,FASc
Biostatistics Consultant, Max Healthcare

2. AGREEMENT
Quantitative agreement between two measurements –
whether one can replace the other
Blood pressure measured by mercury
sphygmomanometer and electronic hand-held
monitor
Enzymes in saliva and blood
Current method: Bland-Altman method of agreement
New method: Direct use of clinical tolerance limits

3. BLAND–ALTMAN METHOD OF
AGREEMENT
Calculate Bland–Altman limits: 𝒅 − 𝟐𝒔𝒅, 𝒅 + 𝟐𝒔𝒅
where 𝒅 = 𝒙 − 𝒚
If these limits are within clinical tolerance, agreement exists,
otherwise not
Clinical tolerance – Minor difference between two
measurements that can not alter the management of the patient
(such as 3 mmHg in syst. BP)
More than 35,000 results in Google search more than 10,000
documents in PubMed for this method

4. LIMITATIONS OF BLAND–ALTMAN LIMITS
Require pre-set level of confidence (95%)
Require Gaussian distribution
Sensitive to outliers (distorted) - mean and SD
Always symmetrical with respect to 𝒅
Both the limits : 𝒅 − 𝟐𝒔𝒅, 𝒅 + 𝟐𝒔𝒅 are subject to
sampling error and need their confidence limits
How to overcome these problem?

5. In any case, clinical tolerance limits necessarily
required for B-A method to work
Expected to be pre-specified
Can we directly use clinical tolerance limits for
assessing agreement?
YES, and all the problems stated previously are also
addressed
CLINICAL TOLERANCE LIMITS

6. AGREEMENT BY CLINICAL TOLERANCE LIMITS
Set clinical tolerance limits as for the interpretation of the B-A method
Find the percentage of differences within these limits
This directly gives the percentage agreement - if needed, find the
lower bound by Wilson method to get the minimum agreement
If a cutoff needed, use 90%
No need of B-A limits:
Extremely simple method
No need of Gaussian distribution
No need to worry about outliers
No need to get CI of lower and upper limits

7. EXAMPLE
Fasting blood glucose level
- measured by the conventional venous method
(autoanalyzer)
- measured by finger prick (capillary by glucometer)
Capillary level is known to be higher than venous level
A company has devised an improvised glucometer that
gives readings after adjusting for higher capillary values

8. NEW GLUCOMETER
Company’s Claim: New glucometer readings can still be
higher despite adjustment but by not more than 5
mg/dL in at least 90% cases
Clinicians willing to accept this much error due to
distinct advantage of capillary sampling – a difference of
5 mg/dL will not make any difference in the clinical
management of the patient
Expected random variation is ±2 mg/dL
Thus, the clinical tolerance limits (-2, +5) mg/dL
Asymmetric limits - B-A method does not allow but our
method allows

9. DATA (n = 40)
Subject No.
Fasting blood glucose level (mg/dL)
Difference
(mg/dL)
Percentage
difference
(%)
Method-1 Method-2
1 106 110 4 3.77
2 82 80 -2 -2.44
3 121 126 5 4.13
4 95 97 2 2.11
5 178 199 21 11.80
6 147 145 -2 -1.36
7 135 138 3 2.22
8 140 139 -1 -0.71
9 112 115 3 2.68
10 126 130 4 3.17
11 130 129 -1 -0.77
12 106 105 -1 -0.94
13 187 195 8 4.28
14 77 80 3 3.90
15 120 124 4 3.33
16 118 121 3 2.54
17 67 65 -2 -2.99
18 136 141 5 3.68
19 98 99 1 1.02
20 102 105 3 2.94
21 118 121 3 2.54
22 182 180 -2 -1.10
23 167 160 -7 -4.19
24 132 135 3 2.27
25 82 82 0 0.00
26 79 80 1 1.27
27 139 138 -1 -0.72
28 125 127 2 1.60
29 119 118 -1 -0.84
30 78 83 5 6.41
31 131 132 1 0.76
32 145 143 -2 -1.38
33 169 172 3 1.78
34 158 157 -1 -0.63
35 144 145 1 0.69
36 138 137 -1 -0.72
37 121 131 10 8.26
38 107 106 -1 -0.93
39 125 127 2 1.60
40 138 142 4 2.90

10. BLAND-ALTMAN METHOD
𝒅 = 𝟏. 𝟗𝟖 𝒔𝒅 = 𝟒. 𝟑𝟗 mg/dL
B-A limits of agreement
(-6.81, 10.76) mg/dL
Clinicians find them too wide
(beyond clinical tolerance)
Agreement not adequate
If the outlier is excluded, the
limits are (-4.85, +7.83) – still
beyond clinical tolerance.

11. Percent agreement (differences within -2 and
+5 mg/dL) = 90%
Sufficient agreement as per the company’s
claim
Wilson lower bound 81% (likely minimum
agreement)
No need of Gaussian distribution
No need to consider CI of upper and lower
limits
No need of symmetrical limits
Not affected by outliers
If one wishes to add a condition that no
difference should exceed 2 times the upper
limit (2x5=10 mg/dL) in this cases, our method
allows and tells that one value is beyond this
limit
DIRECT METHOD

12. PROPORTIONAL DIFFERENCES - I
Difference between venous and capillary values
of FBS may increase as the values increase
- The difference between 120 and 126 (d=6)
mg/dL is 5% and the difference between 300
and 315 (d=15) is also 5%
Can we check for agreement within proportional
difference?
YES, with our method but not with B-A method
(unless we take log)

13. Consider ±2% (Random variation) – Clinical tolerance
Now the plot is as follows:
Only 19 out of 40 (47.5%) difference are within this
clinical tolerance - low and not acceptable
PROPORTIONAL DIFFERENCES - II

14. CONCLUSION
Direct use of clinical tolerance limits is far better
method for assessing agreement than the prevalent
Bland–Altman method
Ref: Direct use of clinical tolerance limits is far better
method for assessing agreement: A robust non-
parametric approach.
Preprint available at
https://www.preprints.org/manuscript/202108.0343/v1