includes various stages and criterion of development and validation of assays. optimisation standardistation specificity sensitivity robustness ruggedness thresold cutoff and proficiency of tests

1. Development and Validation of Diagnostic Assay’s
By:- Ritik Kaul (2022V16M)
M.V.Sc Scholar
Department of Veterinary Medicine
1

2. INTRODUCTION
ASSAY
An assay is an investigative procedure for qualitatively assessing or quantitatively
measuring the presence, amount or functional activity of a target entity (analyte).
VALIDATION
• It is a process that determines the fitness of an assay, for an intended purpose, in
the intended species. It includes analytical and diagnostic performance of the test.
• A validated assay consistently provides test results that identify animals as positive
or negative for an analyte or process (e.g. antibody, antigen, or induration at skin
test site)
• By inference, accurately predicts the infection status of animals with a
predetermined degree of statistical certainty.
2

3. Variables that affect the assay performance can be grouped into three categories
 The sample — host/organism interactions affecting the analyte composition and
concentration in the serum sample.
• Inherent Factors (e.g. age, sex, breed, nutritional status, pregnancy, immunological
responsiveness).
• Acquired Factors (e.g. passively acquired antibody, active immunity elicited by
vaccination or infection).
• Non-host factors, such as contamination or deterioration of the sample, may also
affect the analyte in the sample.
 The assay system —Instrumentation and technician error, reagent choice and calibration,
accuracy of controls, reaction vessels, water quality, pH and ionicity of buffers and diluents,
incubation temperatures and durations, and error introduced by detection of closely related
analytes, such as antibody to cross-reactive organisms.
 The test result —Diagnostic sensitivity (D-SN), Diagnostic specificity (D-SP), and
Prevalence of the disease in the population targeted by the assay.
3

4. STAGES OF ASSAY DEVELOPMENT &VALIDATION
4

5. Criteria for assay development and validation
1. Fitness for intended purpose
2. Optimization
3. Standardization
4. Robustness
5. Repeatability
6. Analytical sensitivity
7. Analytical specificity
8. Thresholds (cut-offs)
9. Diagnostic sensitivity
10. Diagnostic specificity
11. Reproducibility
12. Ruggedness
5

6. Fitness for intended purpose
The purpose of development of an assay may be :
1) Demonstration of freedom from infection inner country free with or
without vaccination or freedom after outbreaks
2) Freedom from infection for the purpose of trade/ movement hey
animals or their products
3) Eradication or elimination of infection from a region or country
4) Confirmation of diagnosis
5) Study of prevalence of infection or risk analysis
6) Determination of immune status (post-vaccination) of animals/
population
6

7. OPTIMIZATION
• Optimization is the process of evaluation of physical, chemical and
biological parameters of an assay to ensure that the assay is precisely
as per the intended application.
• The process of optimization is labour intensive and critical for a
quality assay
• For optimization process at least three well defined reference samples
ranging from negative low and high positive representing known
uninfected and infected animals from the target population should be
selected.
7

8. STANDARIZATION
• Standardisation constitutes calibration of an assay to standard international
national or in house enlightened reference reagents.
• The International Reference Standards containing a known concentration
of target analyte prepared and distributed by international reference
laboratories are the reagents to which all the assays should be standardised.
• In case an international reference standard is not available a national
reference standard can be used as a standard for this assay.
• The National Reference Standards are prepared and distributed by a
national reference laboratory by calibrating with an international standard
reagent.
• These tenders are highly characterised reagents prepared through extensive
analysis with the pair reviewed published methods for their
characterization preparation storage etc.
8

9. ROBUSTNESS
• Robustness is the capacity of an essay to remain unaffected by small
variations in the test situations and is assessed by introducing deliberate
variations in method parameters.
• During checking for robustness of an essay if slight variation in conditions
are reagent concentrations causes unacceptable variability the assay is just
as not to be robust.
• Accordingly a decision may be taken right at this point whether to continue
with validation of the assay or not the most common factors affecting the
robustness of an assay may be quantitative (pH, molarity temperature etc.)
or qualitative reagents microtitre plates, aqueous or organic matrix).
• Likewise for ligand binding essays the lack of robustness may be due to
less concentration of the bio reagent or due to the intrinsic characteristics
of the biological reagent that is whether monoclonal polyclonal or affinity
purified antibody.
9

10. REPEATABILITY
 Agreement between results of replicates of sample votes within and between runs in a
laboratory and is estimated by evaluating variation in results of replicates. The variation
between run is measured by using the same samples in multiple runs and may be expressed as
standard deviation confidence interval etc.
 For ELISA, raw absorbance values are usually used at this stage of validation - Three-to-four
replicates of each sample is run in at least five plates, on five separate occasions are sufficient to
provide preliminary estimates of repeatability.
 Coefficients of variation (standard deviation of replicates/mean of replicates), generally less
than 20% for raw absorbance values, indicates adequate repeatability, if excessive variation
(>30%) is apparent more preliminary studies should be done.
 An assay that is inherently variable, has a high probability of not withstanding the rigors of day-
to-day testing on samples from the targeted population of animals.
10

11. Determination of analytical sensitivity and specificity
 Analytical sensitivity - the smallest detectable amount of the target analyte - assessed
by end-point dilution analysis, which indicates the dilution of serum in which antibody
is no longer detectable.
 Analytical sensitivity is different for direct and indirect detection assays
 For direct assays the analytical sensitivity is most commonly expressed as the number
of copies (genome) or units (complement fixing, colony forming unit, plaque forming
unit etc.) of the target agent to be detected.
 For indirect detection assays the analytical sensitivity is estimated as the least amount
of antibody detected in comparison to the matrix control
11

12. Analytical specificity –
Degree to which the assay distinguishes between the target analyte and other
components that may be detected in the assay in three distinctive ways
namely selectivity, exclusivity and inclusivity of the assay.
• Selectivity refers to the extent of an essay for accurately quantifying the
targeted analyte in the presence of interferents (such as enzyme inhibitors)
degradants (example toxic) factors nonspecific binding (absorption to the
wall of microtiter plate). Further the selectivity also differentiates between
the antibodies as a result of vaccination and active infection that is DIVA
strategy.
• Exclusivity is the ability of an assay to detect the unique antigen or genomic
sequence of the targeted organism excluding other potentially cross reactive
organisms.
• Inclusivity is the capacity of an assay for detection of several strains or zero
bars of species, species of a genus or similar grouping. Inclusivity defines
the scope of an action of a screening assay.
12

13. THRESHOLD
• The threshold or cut off is the test value that distinguishes positive and
negative results for test values between the positive and negative/ cut
off zones various synonyms are used such as intermediate,
inconclusive, suspicious etc.
• For determination of threshold of an essay the test results first must be
categorised to positive negative or intermediate status by insertion of 1
or 2 cut off points (threshold or decision limits)reflecting the intended
purpose of the assay.
• If considerable overlap occurs in their distributions of the test values
two cut offs can be selected and the values falling in between may be
classified as intermediate
13

14. Diagnostic sensitivity and specificity
Principles and definitions :
• Diagnostic sensitivity (D-SN) is the proportion of known infected reference animals that test
positive in the assay; infected animals that test negative are considered to have false
negative results.
• Diagnostic specificity (D-SP) is the proportion of uninfected reference animals that test
negative in the assay; uninfected reference animals that test positive are considered to have
false positive results.
• Sometimes it is difficult to select a cut-off that will accurately classify these animals according
to their infection status. Rather than a single cut-off, two cut-offs can be selected that define a
high D-SN (e.g. inclusion of 99% of the values from infected animals), and a high D-SP
(e.g.99% of the values from uninfected animals).
If the feasibility and initial development & standardization studies indicate that the assay
has potential for field application, the next step is to identify the assay's performance characteristics.
14

15. Calculation of diagnostic sensitivity and specificity
 The selection of a cut-off allows classification of test results into positive or negative categories.
 Calculation of D-SN and D-SP are aided by associating the positive/negative categorical data with the
known infection status for each animal using a two-way (2 x 2) table.
 After the cut-off is established, results of tests on
standard sera can be classified as true positive
(TP) or true negative (TN) if they are in
agreement with those of the gold standard (or
other standard of comparison). Alternatively,
they are classified as false positive (FP) or false
negative (FN) if they disagree with the standard.
 Formula for D-SN = TP/(TP+FN) and D-SP =
TN/(TN+FP) & results expressed in
percentages.
15

16. • It is the ability of an assay to provide consistent results of the same
samples when tested in different laboratories (located in different
regions or countries) using the identical assay protocol reagents and
controls.
• Reproducibility is an important measure of precision of any test
method.
• A minimum three laboratories should test the identical panel of
samples (at least 20 samples by each laboratory)
16

17. Interpretation of test results - factors affecting assay validity
 An assay's test results are useful only if the inferences made from them are accurate, A common
error is to assume that an assay with 99% D-SN and 99% D-SP will generate one false positive and
one false-negative result for approximately every 100tests on animals from the target population.
 Such an assay may be precise and accurate, but produce test results that do not accurately predict
infection status.
 For example, if the prevalence of disease in a population targeted by the assay is only 1 per 1,000
animals, and the false positive test rate is 1 per 100 animals (99% D-SP), for every 1,000 tests on
that population, ten will be false positive and one will be true positive.
 Hence, only approximately 9% of positive test results will accurately predict the infection status of
the animal; the test result will be wrong 91% of the time.
 This illustrates that the capacity of a positive or negative test result to predict infection status is
dependent on the prevalence of the infection in the target population. 17

18. Interpretation of test results - factors affecting assay validity
 An estimate of prevalence in the target population is necessary for calculation of the predictive
values of positive (PPV) or negative (NPV) test results.
 It is highly desirable to provide an interpretation statement with test results accompanied by a
small table indicating PPV and NPV for a range of expected prevalence of infection in the
target population.
 Without provision of such information, test results from the assay may have failed to accurately
classify the infection status of animals, and thus do not reflect a fully validated assay.
18

19.  A validated assay needs constant monitoring and maintenance to retain that designation
 Reproducibility between laboratories should be assessed at least twice each year.
 Proficiency testing is a form of external quality control for an assay – usually administered by a reference
laboratory that distributes panels of samples, receives the results from the laboratories, analyses the data,
and reports the results back to the laboratories. If results from an assay at a given laboratory remain
within acceptable limits and show evidence of accuracy and reproducibility.
 Panels of sera for proficiency testing should contain a full representation of an analyte's concentration in
animals of the target population.
 When the assay is to be transferred to a completely different geographic region, it is essential to re-
validate the assay by subjecting it to sera from populations of animals that reside under local conditions.
 When serum control samples are nearing depletion, it is essential to prepare and repeatedly test the
replacement samples. When other reagents, such as antigen for capture of antibody, must be replaced,
they should be produced using the same criteria as for the original reagents, and tested in at least five runs
of the assay. 19

20. 20