This presentation discusses in a comprehensive way Lab method verification and all related concepts and measures

1. Laboratory Method Verification
Ola H. Elgaddar
MD, PhD, MBA, CPHQ, LSSGB,
Lecturer of Chemical Pathology
Alexandria University
Ola.elgaddar@alexu.edu.eg

2. Quality
Doing the right thing right, from the
first time and every time!

3. Accreditation
An institution or a program meets
standards of quality set forth by an
accrediting agency

5. Validation / Verification??
Method validation:(Manufacturer concern)
Establishing the performance of a new
diagnostic tool.
Confirmation, through the provision of
objective evidence, that the requirements for
a specific intended use or application have
been fulfilled’ (doing correct test)......
ISO 9001:2005

6. Validation / Verification??
Method verification: (Lab / user concern)
A process to determine performance
characteristics before a test system is utilized
for patient testing.
Confirmation, through the provision of
objective evidence, that specified
requirements have been fulfilled’ (doing test
correctly)……ISO 9001:2005

8. According to Westgard
Ø The inner, hidden, deeper, secret meaning
of method validation is error assessment.
ØHow much error might be present in the
test result within your laboratory ?
ØCould this degree of error affect the
interpretation and possibly patient care ?
If the potential error is large enough to lead to
misinterpretation, then the method is not
acceptable.

10. Ø Random error, RE, or imprecision is
described as an error that can be either
positive or negative, whose direction and
exact magnitude cannot be predicted, where
the distribution of results when replicate
measurements are made on a single
specimen.
ØUsually, due to error in Pipetting

12. Ø Systematic error, SE, or inaccuracy is an
error that is always in one direction,
displacing the mean of the distribution from
its original value.
ØIn contrast to random errors, systematic
errors are in one direction and cause all the
test results to be either high or low.
ØEither constant or proportionate
ØUsually, due to error in calibration

16. Internal Quality Control (IQC) is used, on daily
basis, in the decision to accept or reject
results of patients samples and enables the
lab to describe and monitor the quality of its
work.
-Usually it has 2 levels (Sometimes 3);
representing the “Normal” and the
“Pathological” analyte level.
- Judged according to Westgard Multi-QC
rules

17. External Quality Control (EQC) =
Proficiency test is used, on monthly or Bi-
weekly (Or others) basis, where labs from all
over the world join the program and send their
used Method / Analyzer.
- A statistical comparison is made and each
lab result is compared to the result of its peer
group in each analyte.

19. The following items need verification
ØAnalytical Specificity: Interference studies
ØAnalytical Sensitivity: Calibration curve
Detection limit
ØReportable range: Linearity experiment
ØPrecision: Replication study
ØAccuracy: Bias / Recovery study
ØReference Intervals

21. Analytical Specificity
The ability of an analytical method to detect
“ONLY” the analyte of interest.
Freedom from interference by any element
or compound other than the analyte of
interest

23. Analytical Specificity
- Analytical Specificity is verified using
interference studies.
- A validated method, known to be free of
the interfering substance is used. A series
of samples containing increased
concentrations of the interfering substance
are analyzed using that method, and the
method under study, then both results are
compared

25. Some Automated systems have a “HIL” index

27. Analytical Sensitivity
- The ability of an analytical method to
detect a low concentration of a given
substance in a biological sample. The lower
the detectable concentration, the greater
the analytical sensitivity.
- Detection limits studies

28. Analytical Sensitivity
OR,
-The ability of an analytical method to
detect (respond to) a change in
concentration of the analyte. The smaller
the detectable change (the change in
concentration that can result in a definite
change in the reported signal), the greater
is the analytical sensitivity.
- Slope of the Calibration Curve

30. REMEMBER!!

31. STANDARD
• It is a solution of known concentration.
• Formed by dissolving a known amount of
an analyte in a specific volume of an
aqueous solvent.
• Ex: Dissolving 100 mg glucose in 100 ml
D.W gives a standard of a 100 mg / dl
concentration

32. CALIBRATOR
• A solution or a device of known
quantitative or qualitative characteristics
(eg: concentration, activity, intensity)
• Used to calibrate or adjust a measurement
procedure.
• Matrix is preserved.

33. • The calibrator material is reconstituted &
introduced to the analyzer before using a
new lot of reagent, and its concentration is
assigned.
• The calibrator is treated like samples and
the absorbance of the developed color is
determined.

34. Using the provided concentration, the analyzer
constructs a calibration curve
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0 50 100 150 200 250

35. • If the given concentration is 100 mg / dl,
the absorbance of this point is determined.
• By providing the analyzer with the range
where this method is linear, the calibration
curve is extended so that any sample
concentration lying within that linearity
range can be deduced from this curve

36. Detection Limits
1.Lower Limit of Detection (LLD)= Limit of
the blank (LOB)
2.Biologic Limit of Detection (BLD)= Limit of
Detection (LOD)
3.Functional Sensitivity (FS)= Limit of
Quantification (LOQ)

38. Detection Limits
1.Lower Limit of Detection (LLD)= Limit
of the blank (LOB)
Most analytical instruments produce a signal even
when a blank (reagent without analyte) is
analyzed. This signal is referred to as the noise
level. The LLD is the analyte concentration that is
required to produce a signal greater than two (or
three) times the standard deviation of the noise
level.

39. Detection Limits
1.Lower Limit of Detection (LLD)= Limit
of the blank (LOB)
Limit of Detection (LLD) is estimated as, the mean
of the blank sample plus 2 or 3 times the SD
obtained on the blank sample:
LLD = mean(blk) + Zs(blk)
where the Z-value is usually 2 or 3

40. Detection Limits
2.Biologic Limit of Detection (BLD)= Limit
of Detection (LOD)
-It is the limit of blank, after the addition of the
analyte of interest (Its lowest concentration)
- BLD is estimated as the LLD plus 2 or 3 times the
standard deviation obtained from the "spiked"
sample
BLD = LLD+ Zs(spk)
where the Z-value is usually 2 or 3
- Results between BLD & LLD should be reported
without quantitation

41. Detection Limits
3.Functional Sensitivity (FS)= Limit of
Quantification (LOQ)
-The lowest concentration of target compounds
that can be quantified confidently, that meets some
pre-specified targets of imprecision, commonly
CV=20%
- Several spiked concentrations must be studied to
determine the precision profile at the low
concentration range and to select the lowest
concentration at which a 20% CV is obtained.

43. To Sum Up!
1.LoB is the highest apparent analyte
concentration expected to be found when
replicates of a blank sample containing no
analyte are tested.
2.LoD is the lowest analyte concentration likely to
be reliably distinguished from the LoB and at
which detection is feasible.
3.LoQ is the lowest concentration at which the
analyte can not only be reliably detected, but at
which some predefined goals are met. The LoQ
may be equivalent to the LoD or it could be at
a much higher concentration.

45. Reportable Range
=
Analytical Measurement Range
It is the range of numeric results a method
can produce without any special specimen
pre-treatment, such as dilution.

46. Ø It should be verified, for the manufacturer’s
claim, using the linearity experiment.
Ø It is performed using either calibrators,
proficiency samples, or samples
Ø Serial dilutions will be made covering the
whole analytical measurement range, and
reaching as close as possible to the claimed
values of the manufacturer.
ØEach dilution is to be processed in duplicate
to remove the element of imprecision.

47. Ø The observed measures (on the X-axis) are
plotted against the expected measures (on
the Y-axis), and a line point to point graph is
constructed for each analyte.
ØThe line is judged visually for its linearity
and according to each experiment, the
analytical measurement range of each
method is verified, where any patient result
obtained in the future, outside the verified
range, cannot be released without further
processing (Dilution or concentration).

48. Dilution Expected conc Observed conc
D1 8.27 8.27
D2 4.56 4.7
D4 2.5 2.6
D6 1.5 1.52
D7 1 1.07
D8 0.5 0.68
Correlation: r 0.999886844
Analytical range: fT4 : 0.1 - 12 ng / dl

49. 0
1
2
3
4
5
6
7
8
9
0 2 4 6 8 10
Observed measure
Expectedmeasure

53. Precision
Closeness of agreement between quantity
values obtained by replicate measurements
of a quantity, under specified conditions.

55. Precision
ØPrecision should be assessed using
quality control material (A minimum of 2
levels), or pooled serum (Of minimum two
concentration levels)
ØEach level of the QC material / pooled
serum is measured 5 times per day (Within
run), for 5 days (in between runs)

56. Precision
ØThe measures obtained from this precision
study are to be collected, and the mean, SD,
and CV are calculated for each parameter,
for the used QC levels / pooled serum.
ØThe obtained CVs, are statistically
compared to the manufacturer’s claim using
ANOVA test of significance, to determine if
there is a significant different between the
obtained CVs (and their verification
intervals), and the manufacturer’s claim at a
certain CI; usually 95%

58. Accuracy
Closeness of the agreement between the
result of a measurement and a true value of
the measurand.

59. Trueness
Closeness of the agreement between the
replicates of result of a measurement, and
a true value of the measurand.

60. ØThe difference between the mean of
replicates of a measurement, and its true
value is the BIAS.
Ø The CLSI calculation of bias is based on
the results of 7 – 11 PT samples; each is
measured in duplicate, and then compared
to the true value (Peer’s mean) using
student T-test.

62. A concept!!!

63. A concept!!!

64. A concept!!!

65. Why is Z-score used in PT results??

66. Bias is verified for the tested method when
there is no significant difference between:
q Mean Z-score of PT results (7 – 11) is not
significantly different from Zero
q Mean Z-score of PT results (7 – 11) is not
significantly different from peer’s mean

68. Reference intervals
ØRemember that we are just “Verifying” the
reference intervals stated by the
manufacturer or published in the literature,
and “transferring” them to the lab using the
method under study.
Ø “Establishment” of reference intervals is
another issue.

71. Reference intervals
ØAcceptability of the transfer shall be
assessed by examining 20 reference
individuals, from our subject population, and
comparing the obtained test results to those
of the manufacturer/ Literature.
ØThose 20 individuals should be selected in
such a way that will satisfy the exclusion and
partitioning criteria.
ØThe test results should be examined to
make sure that none of the results appears
to be an outlier.

72. Reference intervals
ØThe manufacturer's / Literature reference
intervals are considered verified if no more
than two of the 20 tested subjects' values (or
10% of the test results) fall outside those
ranges.

73. Reference intervals
ØExclusion / partitioning criteria include:
age, sex, fasting status, disease history,
drug history, previous surgeries, and time of
the cycle / pregnancy for females.

76. Method Comparison
Ø According to CLSI, at least 40 samples
should be assayed on both methods under
examination (two field methods), or between
one tested method and a reference method.
Ø Several statistical approaches can be
used, one of them is to calculate the
correlation coefficient “r”
Ø“r” should be more than or equal 0.95

78. Total Error
It is the summation of both Random and
Systematic error.
It is calculated as follow:
TE = Bias (%) + 2 CV
It is compared to Biological Variation (or any
other specifications) for Total Allowable
Error

80. Uncertainty
Ø It is an interval around a reported
laboratory result that specifies the location of
the true value with a given probability
Ø It takes into consideration both the
imprecision (SD), and the inaccuracy (Bias)
Ø It is calculated from the data of 6 month
minimum

81. What performance characteristics
are usually validated?
ØReportable range (Linearity)
ØPrecision (or imprecision)
ØAccuracy (or inaccuracy, bias)
ØReference interval