This document discusses the importance of validation for point-of-care testing (POCT) devices. It notes that while POCT allows for quick results, the environment is less controlled than a laboratory and results carry a higher risk of inaccuracy. It then outlines several key steps in validating a new POCT device, including establishing clinical need, choosing the correct device, deciding on sample types, parameters to assess like imprecision and accuracy, and who should perform the validation. The document emphasizes that validation in both laboratory and end-user settings is important to ensure a POCT device performs as intended when used outside the lab.

1. POCT: Wriggly Fish in a Jar
Rajendra Dev Bhatt (PhD Scholar)
Asst. Professor
Clinical Biochemistry & Laboratory Medicine
Fellow: Translational Research (2018-2022) in CVD in Nepal, NHLBI &
NIH, USA

2. POCT devices are appealing in clinical practice,
because they allow quick diagnosis or clinical
decisions based on information received in real
time, which may subsequently have an influence
on therapeutic pathways.

3. POC testing environment is generally less
controlled than laboratory conditions and the
results can be at higher risk of external
interference than laboratory processes, which
can lead to inaccuracy.

4. clear as mud

5. Validation of POCT:
The elephant in the room

6. Validation
The goal of analytical measurement is to gather
precise, accurate, and consistent data
These qualities must be evaluated based on the
findings of method/parameter verification and
validation, which has long been a crucial
component of excellent analytical practice.
In this aspect method verification/validation is an
important requirement in the practice of testing

7. Validation is a technique that offers unbiased proof
for a particular intended application of the method.
In the context of a clinical laboratory, the validation
of an instrument refers to the process of establishing
and documenting evidence that the instrument
consistently produces accurate and reliable results
within specified conditions.
Validation….

8. Validation is the process of evaluating a system or
component during or at the end of the development
process to determine whether it satisfies the
specified requirements.
Validation involves assessing the overall
functionality, usability, and fitness for the intended
use of the POCT.
Validation….

9. Validation….
There are several steps applied to any new
laboratory analytical methodology as part of formal
acceptance testing, and are equally applicable to
POCT to verify the manufacturer performance
specifications of the method.

10. Validation….
Manufacturers are legally required to provide
performance requirements for in vitro
diagnostics, and validation / verification of these
specifications is necessary for laboratories to
receive full accreditation.

11. Validation….
It is questionable if only demonstrating compliance
with predetermined requirements is sufficient to
validate a new test?
A POCT device can be validated in a variety of
ways, ranging from a quick look at the device to a
more in-depth look at how well it works in clinical
and analytical settings.

12. Validation….
Validation plans for POCT devices should be
completed before implementation with involvement
of local laboratories as stipulated by local
accreditation bodies.
Therefore, determining that a clinical need exists
and that the integration of POCT devices into a
diagnostic & therapeutic route is justified. This is
one of the first stages in putting these devices into
use.

13. Method Validation
The presentation of objective proof that a given
POCT device complies with predetermined
standards is known as method verification, a
word that is sometimes used interchangeably
with the term "validation."

14. Method Validation….
These two procedures (validation and
verification)work together to support the
evaluation of a technique by defining the
characters of analytical and clinical performance
to ensure that they satisfy predetermined
standards.

15. Method Validation….
When it comes to POCT devices, validation is
probably done after the device has been approved
and put on the market.
In most cases, commercial POCT methods will
already have been validated by the manufacturer.
The user (a laboratory in most cases), however,
must independently verify that these characteristics
can be reproduced locally.

16. Why is validation necessary?
This is to guarantee, that the performance criteria
specified by the manufacturer are satisfied.
However, validation has a much bigger
responsibility because it shows real-world
performance for the first time, made by end users in
the selected clinical scenario and place.

17. Why is validation necessary?
It is important to note that the published studies
validating the POCT procedures are not necessarily
conducted in the situations in which they will
ultimately be utilized and may be carried out by
trained laboratory experts rather than end users.

18. Why is validation necessary?
For a variety of causes, ranging from reagent
instability and storage concerns to user skill and a
lack of quality control and quality assurance, the
performance of assays may not meet manufacturer-
stated benchmarks.

19. Why is validation necessary?
The validation of POCT technologies is arguably
even more important as POCT devices have
features that make them vulnerable to poor
performance, these may includes:
Compromises to the quality of the analytical
methodology (in some cases)
Pre-analytical variability: change in sample
matrix and patient condition

20. Why is validation necessary?
Specimen, reagent and control degradation
Less-controlled user environment (outside the
laboratory)
Variety of users with wide spectrum of
credentials, backgrounds and experience
Inadequate and inconsistent quality maintenance
(internal quality control and external quality
assessment)

21. New POCT device measuring analyte X
Establish clinical need for POCT device
Decision to procure
device and implement
Within laboratory
validation, undertaken
by trained staff
Outside laboratory
validation
undertaken by end
users
 Does the device meet pre-specified
manufacturer analytical performance
characteristics?
 Are these in line with internationally
agreed characteristics for analyte X?
 What is the relationship between POCT
analyte X results and lab assay results?
 Is the relationship/deviation acceptable?
 Could changes to the laboratory service address
the need?
 What effect on the clinical pathway would POCT
testing of analyte X have over lab testing?
 Is the introduction of POCT X cost-effective?
 Is the use of POCT X supported by national
guidelines?
How does POCT X analytical
method perform in end user hands
and user locations? • Is this within
acceptable limits?
Clinical evaluation
Implementation supported with appropriate
laboratory and clinical guidelines on when to
use POCT X over laboratory X, its limitations
and appropriate to clinical context
recommendations.
Components of an ideal POCT evaluation

23. Magnitude and Scope of Method Validation
A variety of national and international guidelines
have been formulated to support method validation.
Requirements for the components of a method
validation may vary between countries depending
on the level of approval that has been obtained.
For example, in the United States, Food and Drug
Administration (FDA)-approved devices must meet
Clinical Laboratory Improvement Amendments
(CLIAs) requirements for method validation

24. Magnitude and Scope of Method Validation
Some POCT devices that the FDA has judged to
be of low complexity may be CLIA-waived,
allowing areas to only use these low complexity
tests.
However, it is not always clear if CLIA-waived
devices meet the quality specification for the
analytes being measured.

25. Steps to undertake for a method validation
1. Establishing a clinical need:
Priority one in assessing any POCT should be
determining the clinical need for the device.
Technological advances allow for an increasing
number of tests to be feasible at the point of care.
When there is technology push, (if an unmet
clinical need), consideration will need to be given
to whether the device has market potential before
investing.

26. Steps to undertake for a method validation
2.Choosing the correct device for the clinical
application:
After deciding to use POCT for a particular
therapeutic/diagnostic goal, a choice must be taken
on which device to order.
This process is initiated by setting out the basic
requirements and assessing which devices meet
user’s specification.

27. Steps to undertake for a method validation
Considerations may include performance of the
device in published studies, potential interferences,
usability, and provision of connectivity to
electronic patient records.
In some cases, in which there is insufficient
evidence from published studies as to the
performance of POCT devices, a laboratory may
choose to evaluate two or more of the available
devices

28. Steps to undertake for a method validation
3.Decide which samples to test:
Choosing the correct samples to validate the method
is a key step as validation parameters may vary
according to the matrix used.
It is often preferable to use the same matrix to
validate the test as will be used for actual POCT
measurements; however, this may not always be
possible.

29. Sample types that can be used in method
validation
Examples
Same matrix as the device will use Urine, capillary whole blood, whole
blood, serum or plasma from healthy
volunteers or patients of interest
Different matrix if same matrix unfeasible Ethylenediamine tetraacetic acid (EDTA)
whole blood instead of capillary whole
blood from healthy volunteers or patients
of interest
Manufacturer-provided control material High, medium and low controls or
standards
Alternative manufacturer controls High, medium and local control or
standards from a different kit
Certified reference materials If available, certified control / standards of
the measured analyte

30. Steps to undertake for a method validation
4. Who should undertake the validation and
where:
Before starting validation trials, the last thing to
think about is who will be evaluating the results.
The reported performance of POCT devices
according to manufacturer-stated references is
seldom poor, but, despite this, significant real-
world variability is reported.

31. Steps to undertake for a method validation
The first part of a method validation, which
assesses analytical performance, is usually
undertaken within the lab by laboratory-trained
professionals
However, subsequent parts of the method
validation process, for example, assessing bias,
may be better undertaken in user hands.

32. Parameters of method validation
A variety of factors can be assessed when undertaking POCT
method validations.
Major parameters Other
• Imprecision Cross-reactivity
• Linearity Lower limit of quantification
• Accuracy and bias
• Lower limit of detection
• Interference studies
• Reference range verification
• Usability

33. Imprecision
Imprecision in a diagnostic laboratory refers to the
variability or lack of consistency in test results
when measuring the same sample repeatedly.
It reflects the degree of random error associated
with the testing process.
Imprecision can arise due to various factors such
as instrument variation, reagent variability,
operator technique, and environmental conditions.

34. Imprecision
Let's say that a laboratory professional takes three
separate measurements of the same blood sample
using the same analyzer.
The results obtained are as follows: 90 mg/dL, 92
mg/dL, and 88 mg/dL. These values demonstrate
some variability or imprecision in the
measurements.

35. Imprecision
To quantify this imprecision, the laboratory
typically calculates parameters such as the
coefficient of variation (CV) or standard deviation
(SD). The CV represents the relative variation in
the test results, while the SD measures the
absolute variability.

36. Imprecision
The CV is calculated by dividing the standard
deviation by the mean and then multiplying by
100 to express it as a percentage. Let's assume
the mean of the three measurements is 90
mg/dL, and the standard deviation is 2 mg/dL.
CV = (2 mg/dL / 90 mg/dL) x 100 = 2.22%
A lower CV indicates higher precision and less
variability, while a higher CV suggests greater
imprecision

37. Linearity
Linearity (analytical range) is assessment of the
range over which results can be obtained without
dilution.
This parameter may be difficult to assess in POCT
methodologies as the traditional way to assess
linearity is through dilution of high-concentration
samples and it may be problematic to dilute common
POCT samples – whole blood for example, without
introducing matrix effects.
However, a high standard could be used as a
surrogate sample

38. Accuracy and bias
Accuracy refers to how close the measured values
are to the true or reference values. In other words,
it assesses the degree of agreement between the
laboratory test results and the actual values that
are known or accepted as true.
Accuracy reflects the absence of systematic error
in the testing process.

39. Accuracy and bias
Bias, on the other hand, refers to a consistent deviation
or systematic error in the test results.
It indicates a constant offset between the measured
values and the true values, regardless of whether they
are overestimations or underestimations.
Bias can arise from factors such as calibration issues,
methodological limitations, or inherent biases in the
testing process.

40. Interference studies
The presence of an interfering substance in the
patient sample, interacting non-specifically with
the assay, may result in an artefactually higher or
lower result.
For immunoassay-based POCT devices or those
using colorimetric detection, the most commonly
studied interferences include: bilirubin, hemolysis
and lipemia.
It may also be necessary to test interfering drugs or
metabolites.

41. Establishing a limit of detection
This may not be possible if there is a qualitative
test. However, for quantitative devices and when a
lower limit cut-off exists, it may be valuable to
assess the limit of detection, which is defined as
the lowest concentration of analyte in a sample that
can be detected.
This is usually assessed by measuring a blank
sample at least 10 times, calculating the standard
deviation of the results and multiplying this by 3.

42. Verification of the reference ranges and
cut-offs
Reference intervals (the range of values measured
within 95% of the healthy population) are difficult
to validate, not standardized and are often
population specific.
Manufacturer reference ranges are ideally verified,
but in practice this may be challenging, and if
another laboratory has undertaken this already and
the reference range is traceable then it may be
practical to accept their verification.

43. Barriers to adoption
The introduction of a novel POCT involves a
variety of stakeholders including patients,
clinicians, laboratory services, regulators,
commissioners and industry.
These stakeholders will have different attitudes,
beliefs and incentives relating to the adoption of a
novel device.

44. Implementation
The successful implementation of POC testing strategies
into clinical practice requires a structured
multidisciplinary framework to ensure clinical
governance.
Most large health-care organizations have institution-
wide POC leads and committees to oversee the use of
POC devices.
Lines of responsibility and accountability need to be in
place and policies reviewed at frequent intervals to
ensure delivery of a high-quality service.

45. Implementation
As a result, the committee's duties include deciding if
POC testing is necessary at a specific location, auditing,
purchasing, preventing the unauthorized acquisition of
POC equipment, providing training, and ensuring that
quality control and quality assessment procedures are
followed.
Training is vital to the successful implementation of POC
testing strategies into clinical practice, and test use should
be restricted to staff whose training and competence has
been established.
Training needs to cover all phases of the testing process
including sample acquisition, device use and appropriate
response to unusual test results.

46. Point of Critical Thinking (POCT)
• How to choose the right equipment
• Location of equipment
• Required space
• Power source and battery life
• Limitations of test
• Storage of reagents and consumables
• Users and competence
• Maintenance

47. Point of Critical Thinking (POCT)
• Validation―Demonstrate the METHOD is
“equivalent” to the reference method (for the
matrices validated) based on defined method
criteria
• Verification―Demonstrate in the hands of the
USER that the method can be performed to
meet the defined method criteria
• Method validation and verification are tools for
a Good Laboratory Practice

48. Thank You