Using reference materials can help clinical laboratories meet validation and verification requirements for molecular diagnostic tests. Credit Valley Hospital uses Horizon Diagnostics' pooled DNA reference standards at a 2.5% mutant allele frequency as a low positive control in their EGFR diagnostic assays. Including this control helps eliminate false positive results and provides a qualitative reference to confidently identify true low level positives. This reduces the risks of false negative or false positive reports, improving patient outcomes by ensuring accurate molecular testing results.

1. Using Reference Materials to Meet Validation and
Verification Requirements

2. 5
Bio-Specimens used in Molecular Diagnostics
 Most clinical tissue samples are preserved in FFPE
 FFPE samples are now being used for molecular diagnostic testing
 FFPE based studies: every specimen is different in terms of % tumor contribution to
the specimen and % mutation contribution to the tumor
• Therapeutic choices are made based upon these results
• False positive and false negative results are detrimental to the patient
FFPE
Sample
Cancer
Patient
DNA Extraction Diagnosis

3. 3
External Quality Assessment Proficiency Testing Scheme - 2014
Only 70% of laboratories passed the proficiency test.
False -negatives and false-positives were the main sources of error.
0
5
10
15
20
25
30
35
40
PercentageofIncorrectResults
EGFR Sample Tested
EGFR Genotyping Errors
External Quality Assessment 2014

4. What is the impact of assay failure
in your laboratory and how do
you monitor for it?
7

5. 5
HDx™ Reference Standards - Precise Allelic FrequenciesReference
Slide
FFPE Reference
Standards
IHC Reference Slides
FISH Reference Slides DNA Reference
Standards
Tumor sample
Diagnosis
Therapy
DNA extraction
GenotypingCytogenetics
Histology

6. Validation:
Cost, Risk and Technical Possibilities…

7. 7
The Validation Balance
“Validation is always a balance
between costs, risks and
technical possibilities”.
(CAN-P-4E: clause 5.4.5.3- Note 3)

8. 8
Verification and Validation Definitions
• Confirmation by examination and provision of
objective evidence that the particular requirements
for a specific intended use are fulfilled.
Verification
(ISO 9000)
• Confirmation by examination and provision of
objective evidence that specified requirements have
been fulfilled.
Validation
(ISO 9000)
• Safe and useful service to clinicians and patients.
• Validation is more than a one time activity and if
used continually can improve the assessment of test
accuracy and performance.
Purpose

9. 9
General Validation Requirements
General Validation Requirements
Test development – protocols and pooling
parameters
Test validation – establish performance parameters
(sensitivity, specificity, reproducibility etc.)
Platform validation – performance and confidence
intervals including software validation

10. 10
Intent Comparison
US - CLIA
• Regulate Laboratories –
limited scope.
• Test used that is not FDA
cleared/approved 
establish performance
characteristics before release
of test results.
• No oversight of clinical
validity.
• Oversight occurs through
biennial survey – after
testing has started.
• Test complexity framework.
US - FDA QSR (Potential)
• Regulate manufactures.
• Premarket clearance
evaluates clinical validity.
• Clinical validity – accuracy
with which the test
identifies, measures or
predicts the presence or
absence of a clinical
condition or predisposition in
a patient.
• Risk based framework.

11. 11
Requirements Comparison
• Regulates LDTs made and used within a single facility.
• Focus on accurate reproducible and reliable tests.
• Requirement for analytical validation prior to use.
• Requirements for proficiency testing.
US - CLIA
• Address clinical validity
• Risk based approach to implementation.
• Process validation.
• Design verification and validation.
• Adverse Event Reporting.
• Device Listing and Pre-market Approval (for some devices).
US - FDA
QSR
(Proposed)

12. Canadian Regulations
• Accreditation of medical laboratories in Canada is regulated by
provincial health authorities, five of them having accreditation
bodies.
• Each of the bodies has developed its own standards implementing
ISO documents which generally follows ISO 15189 (balance of
business and technical assurance).
• Risk based scale to validation based on if the test is developed in
house, uses nonstandard methods, originates from scientific
literature or de novo.
• Requirements for proficiency testing.
• Importance of method validation.
Canada
(CAN-P-4E)

13. 13
FDA QSR Verification and Validation (Sec. 820.30 Design controls)
• Validate device design under
standard operating parameters.
• Design input meets design output.
• Part of design history file.
Design
Verification
• Validate device design through
production units.
• Conform to intended use of product.
• Part of design history file.
Design
Validation

14. 14
Cost, Risk and Technical Possibilities…
• More regulation may mean more cost but there are still opportunities for
innovation.
• There is opportunity for clinical laboratories to work with the FDA and
CLIA to develop an appropriate regulatory framework.
Cost
• The FDA’s plans to regulate LDTs are both risk based and to reduce risk.
• Most accreditation programs logically consider risk in the level of
validation required.
• Risks are changing, many LDT’s are moving beyond their traditional
boundaries.
Risk
• Development of new technologies.
• Changes to the standard of care.
• Increasing availability of reference materials, including HDx reference
materials.
Technical
Possibilities

15. 15
Final Thoughts
Final Thoughts
• Use of reference materials can assist in
validation studies but there is still a strong
focus on use of patient specimens, in spite
of increasingly available reference materials.
• Where reference materials are mentioned,
these focus on cell line DNA over plasmids.
• The future for FDA regulation of LDTs is
being developed. The impact of this
regulation remains for debate.

16. 16
Key Resources
Key Resources
• http://www.nature.com/ejhg/journal/v18/n12/full/ejhg2010101a.html
• http://www.horizondx.com/applications/applying-our-
formats/implementation-and-accreditation-of-iso-15189
• http://www.horizondx.com/applications/applying-our-formats/next-
generation-sequencing-guidelines-new-york-state
• ISO 15189:2007
• ISO 17025:2005
• http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandG
uidance/GuidanceDocuments/UCM416685.pdf

17. Case Study at Credit Valley Hospital
A Discussion About the Value of Including Low Positive DNA
Control in Each Assay

18. 18
Methodology
 EGFR mutation status test: Entrogen commercially available kit
 Instrument: Roche LC480 Real Time PCR
 Allele discriminating assays rely upon the ability of the probes to bind correctly and
the software setting to correctly set a crossing point baseline
 Controls: HDx™ Reference Standards to
 Validate assay and establish detection
 Limits for each mutation

19. Assay Validation and Specific Crossing Points Cut-offs
20%
10%
5%
Percentage Control CP Mutation CP
L861Q
20% 30.35 29.66 30.37 30.58 30.66 30.45
10% 29.58 30.17 31.16 31.63 32.08 32.07
5% 29.50 29.72 31.94 32.82 33.18 33.26
1% 30.22 29.65 29.55 37.48 37.29 36.12
0% 29.39 29.20 30.26 n/a n/a n/a
1%
LOD Validation Curve with HDx™ Reference Standards EGFR L861Q
19

20. Validation Issues: Background Amplification in the G719X Reaction
Mutation Positive Control
Noise band
Patient (Cp 37)
 Negative control Cp can lead to a false positive
result for the patient
 Manually changing the baseline hides quality
issues such as inefficient probe binding or PCR
contamination in the blank
Mutation Negative Control
Mutation Negative Control
Noise band
20
 Inefficient probe binding and background noise is seen most in G719X and T790M

21. Validation Issue: Separation from Background Amplification CP Values
 T790M and G719X negative control and 1% Low Positive control values overlap
 Crossing point value from 2.5% DNA HDx™ Reference Standard for each mutation
provides best separation from background amplification
2.5% Crossing Point Value used as a cut off
Low Positive Control now included in each assay to remove false positives
20.0
22.0
24.0
26.0
28.0
30.0
32.0
34.0
36.0
38.0
40.0
T790M Exon 19del L858R L861Q G719X
2.5%
1%
Neg Ctl
21
CrossingPoint
EGFR Variant

22. Low Positive Control helps identify and correct false positives from Non-
Specific Probe Binding
EGFR G719X mutations were safely ruled out by use of the 2.5% cut-off value.
G719X Reaction
Kit PC (28.24)
2.5% Control (31.85)
Patient A (34.70)
Patient B
Negative Control
NTC

23. Added Value from Inclusion of a Low Positive Control
Inclusion of a 2.5% low positive control allows background
amplification to be ruled out
Background amplification can lead to false positive diagnostic
results
Big Picture
Cancer patients awaiting treatments from diagnostic results
False positives  TKI therapy for true EGFR WT patients
• Proven to be detrimental over first-line chemotherapy (Patton et al., 2014)

24. Diagnostic Test Issue: Low Mutation Content Below Cutoff
Should the 2.5% control crossing point be the absolute cut-off in all
cases?
When mutation is in a low percentage of cells in the specimen, the
crossing point will be high, and possibly above the 2.5% established
cutoff
How should a high crossing point value be evaluated?
Inverse
Relationship
24
Crossing Point (CP) Allelic Frequency %
in Tumor sample

25. Low Positive Control as Absolute Cut Off?
Patient 1: 58 year old female Metastatic lung cancer
• Mutation negative for all mutations except L858R
• L858R crossing point slightly higher than 2.5% control
• Qualitative assessment of amplification curve suggests true positive at about 1-2%
L858R Reaction
Kit PC (29.31)
Patient C (34.41)
2.5% Control (33.58)
Neg Ctl/NTC

26. Low Positive Control as Absolute Cut Off?
Patient 2: 58 Year old female with right lung adenocarcinoma
• Mutation negative except for Exon 19del
• Exon 19del crossing point slightly higher than 2.5% control
• Qualitative assessment suggests this is a true positive, at about 1-2%
Exon 19del Reaction
Kit PC (28.39)
2.5% Control (30.71)
Patient D (33.30)
Neg Ctl/NTC

27. Conclusion – Qualitative and Quantitative QC for each run needed
Patient Outcomes
Qualitative assessment suggests these are true positive results
Patients are receiving anti-EGFR therapy and both report feeling better, less
fatigue, still able to work
Conclusion
Crossing point values alone do not tell the whole story
False negatives can occur from absolute/quantitative use of 2.5% control
• Confidently ruled out by qualitative interpretation of patient amplification curve
False negative  chemotherapy for EGFR mutant patient
• Less effective than TKIs for patients with an EGFR variant (Patton et al, 2014)
Horizon Diagnostics is proud to support clinical laboratories with the provision of sustainable reference materials
for research use only; applications include the validation of equipment, consumables and laboratory developed
tests offline from patient testing, supporting the healthcare continuum for the regulators, payors, clinical
laboratory and ultimately the patient.

28. Analytical Utility of a Low Positive Control
Inclusion of a 2.5% low positive control allows elimination of false positives
Low positive controls supply a qualitative reference for true amplification
so that low positive results can be reported with confidence
Reduce the risk of false negative reports
It a QC check for each run
Provide consistency between patient EGFR mutation status experiments
28
Reduction in False Negatives and False Positive Results is needed for
better future PT results and diagnostic testing

29. 2.5% Crossing Point Values Very Consistent Between Assays
T790M Exon19del L858R L861Q G719X
Kit Pos Ctl 28.13 29.17 29.87 28.73 28.36
2.50% 32.64 31.61 34.1 32.12 31.9
26
27
28
29
30
31
32
33
34
35
36
37
CrossingPoint
Control Crossing Points Comparison
29
95% confidence intervals for 17 different assays
Mean Crossing points of 2.5% Low positive control: +/- 0.26 (L858R) to +/- 0.56 (G719X)

30. The Value of a Low Positive Control at Credit Valley Hospital
Case study of how a Canadian clinical laboratory used RUO products to
validate their laboratory developed test
Credit Valley Hospital includes a 2.5% MAF reference standard for all their
EGFR diagnostic tests
• Individual EGFR Base-Seq DNA HDx™ Reference Standards are ordered and
pooled to a 2.5% mutant allelic frequency
• A Low Positive Control has also been implemented for the BRAF, KRAS and
NRAS testing at this laboratory
This gives them the confidence that their assay is working today, and also a
qualitative benchmark for true amplification when patient results are close
to cut off CT values
30
EGFR Low
Positive
Control
KRAS Low
Positive
Control
BRAF Low
Positive
Control
NRAS Low
Positive
Control

31. 7
“The low positive control is critical to my confidence in the
lab’s diagnostic reporting and analytical results.”
Dr. Marsha Speevak, Discipline Lead
Molecular Genetics and Cytogenetics, Credit Valley Hospital,
Trillium Health Partners
What is the impact of assay failure
in your laboratory and how do
you monitor for it?

32. Assay Plate Set Up – Supplemental Data
 Credit Valley Uses 4 lanes of controls in their kit
1. Kit PC
2. Blank
3. Kit Negative
4. Horizon Dx pooled 2.5% Control