This document discusses developing analytical evaluation standards for next generation sequencing (NGS) diagnostic tests. It outlines the FDA's vision of implementing standards and shared resources to enable research and provide accurate NGS test results for patients. The document discusses developing performance standard elements, areas needing standards, the overall NGS test workflow, and existing guidelines. It also discusses stakeholder groups involved in standard development and a proposed bioinformatics tools repository to help developers meet standards. The success of precision medicine will require accurate and safe NGS diagnostic tests within a learning healthcare system.

1. Analytical Evaluation Standards for NGS
Zivana Tezak, PhD
OIR/ CDRH/FDA
August 28, 2015
NIST, Gaithersburg, MD
Genome in a Bottle Consortium--
August 2015 Workshop

2. 2
Disclaimer
• NGS is raising new policy / regulatory issues;
thoughts presented here are preliminary and do not
represent proposed or finalized FDA policy – these
are just preliminary ideas or approaches we are
considering
Background (“Vision”)
• February 2015 - “Optimizing FDA’s Regulatory
Oversight of Next Generation Sequencing Diagnostic
Tests” http://
www.fda.gov/MedicalDevices/NewsEvents/WorkshopsConferences/ucm427296

3. Precision Medicine Initiative - FDA
Vision: Implement new regulatory policies to promote
research and accelerate the translation of precision
medicine technologies into treatments that benefit
patients.
•Near Term: Implement standards and shared resources that
will enable the development of knowledge for research and
patient decision making
•Longer Term: Implement standards-based regulation of NGS
diagnostic tests that will ensure that the tests patients receive
provide accurate, reproducible, and meaningful results
3

4. FDA – Modernizing Regulation for NGS tests
Adopt a flexible, dynamic
regulatory system
(knowledgebase /
databases)Develop and
implement standards
to assure quality
Develop cloud-based
open-source tools
to help test
developers meet
standards
Analytical evaluation
Bioinformatics
Clinical significance

5. Build growing suite of standards
– Performance standard documents
e.g., ACMG, CLSI, NYS DOH, CAP
– Material standards
e.g., NIST/GIAB
5

6. General Overview / Framework
• Analytical: Performance Standards as a Regulatory Tool for
NGS Testing
– Analytical performance standard elements
– Uses / areas in need of separate analytical standards
– Overall architecture / workflow of clinical NGS test (from sample to
EHR)
– Example of a possible performance standard (single use)
– Existing guidelines and standards for NGS tests
– Stakeholder communities building some of the standards
– Cloud based, open source bioinformatics tools repository to facilitate
standard development
6

7. Performance standard elements
• Intended use of the test (e.g., carrier screening, diagnosis of
inherited disease, cancer risk prediction)
• Transparency
• Test description (methodology, instrumentation, reagents,
software, etc)
• Metrics (quality scores, read depth, sensitivity/specificity, etc)
– what to measure
• Protocols and studies to perform (specimen type, sample
numbers, comparator methods, etc) – approaches on how to
measure analytical performance
• Acceptance criteria, thresholds for measurements
• Performance data / result presentation
7

8. Major areas in need of separate standards
Initial assessments; will develop and change. Where to start?
– Targeted clinical assays
• Germline targeted assays
• Somatic targeted assays (cancer panels)
– Exome-based clinical assays (heritable, rare undiagnosed disease)
– WGS-based clinical assays
• For germline uses
• For somatic uses
– Method or analytic quality-based standards
• for methodology, that would include different classes of variants, genomic regions,
allele ratios, different technology
• for platform (use of standard materials, panels)
• for test building blocks (e.g., preanalytical, library, software, quality system
approach, etc)
– Run quality standards – including computational solutions (WGS only?)
– Terminology (language agreement and definitions)
– Agreement on appropriate comparators and “gold standards”
– Material standards, reference samples, disease-specific sample sets, and
controls
– Data formats and interchangeability / interoperability 8
Byintendeduse

9. Overall architecture of NGS test
• High-level NGS test steps, from patient sample through variant
call and interpretation:
– Preanalytical (from tissue -> DNA); includes specimen quality
assessment, DNA preparation, library preparation
– Sequencing (from DNA -> raw sequence data)
– Sequence bioinformatics (from raw sequence data -> VCF); includes
pipeline from aligning, filtering, read mapping, annotation,
contamination, through call determination
– Interpretation (including knowledgebases)
– Reporting
– EHR and archival
• Will differ for different platforms
• Overall standards architecture?
9

10. NGS Workflow Standards and Tools
Examples of Intended Uses
Hereditary
Diseases
(targeted
panels)
Hereditary
Diseases
(exomes)
Cancer panels Methodology
Preanalytical
Sequencing
Data Processing
(informatics)
Knowledgebase
(databases)
Interpretation
Reporting
NGS Workflow

11. Performance standard example
• Single use
• Possible analytical approach
• Heritable monogenic disorders
– Diagnosis
– Targeted panels
– Exomes?
• Additional examples -
– Special controls for NGS platform and autosomal recessive carrier
screening
11
Need flexibility for evolving technology and knowledge!

12. Existing standards and guidelines
• Currently existing or ongoing standard development, how they
correspond to FDA needs:
– Material standards – e.g., NIST, sample panels
– Methodological standards – e.g., NYSDOH, CAP, MolDx, GA4GH, Nex-
StoCT, ACMG, HGVS, CPIC, CLSI
• What can be used, for what purpose
• Identify gaps
• Questions:
– Who should fill identified gaps
– What is acceptable timeline?
• Need built-in flexibility
12

13. Stakeholder communities
• Stakeholders involved for each area
• Ways to work together
• Possible conveners
– SEQC Consortium
– NIST / GIAB Consortium
– CERSI
– CDC
– CLSI
– GA4GH
– others?
13

14. Bioinformatics - General Landscape
In the NGS-based assay space -
•Some labs offer the test by sequencing the samples but
outsourcing bioinformatics analysis to other companies
•Some labs require support from 3rd party companies to host
their bioinformatics pipeline, but the labs perform the analysis
•Some labs offer the test by sequencing the samples AND have
deep bioinformatics and analytic capability
A
B
C
4

15. FastQ
Community
NGS-Based Test Developers (large and small), NIST,
FDA Scientists, Standards Bodies, Academic Centers,
Patient-Facing Providers, Consortiums
Analysis
and Pipeline(s)
Security and Privacy
HIPAA/HITECH, CAP, ISO27001
Uniquely identified and immutable data
Version-controlled applications
Reference
Data
VCF
Quality
Threshold
Approve
Reject
precisionFDA
Courtesy of Taha Kass-Hout, FDA Chief Health Informatics Officer
Advancing the accuracy and
reproducibility of NGS
•Crowd-sourced, cloud-based
platform
•Will provide tools and open
access resources
•Will allow the community to test,
pilot, and validate approaches to
NGS

16. Community
Development and Use of Analytical Standards
• Current practices
• Technical knowledge
GA4GH
NIST
Others
FDA
Regulatory proposalCompanies
NGS labs
Stakeholder input
Discussion
paper
Public
workshop
Comment
analysis
Standards bodies
Professional
Societies

17. Success of Precision Medicine Requires:
• Safe and accurate diagnostic tests that reliably identify
individual variation
• Learning health systems that enable researchers and
clinicians to learn from and inform the patient experience
• Development of targeted therapies that are more
efficacious or have less deleterious side effects for specific
individuals
• Updated research and regulatory policies that catalyze the
development of new treatments while protecting patients