"Microbial Genomics @NIST" presentation at the Standards for Pathogen Identification via NGS (SPIN) workshop hosted by National Institute for Standards and Technology October 2014 by Nathan Olson from NIST.

1. Nate Olson
Genome Scale Measurements
Biosystems and Biomaterials Division
Microbial Genomics
@ NIST

2. ● Example Microbial and Genomic Programs
○ ERCC spike in controls
○ Genome In a Bottle
○ Biothreat Detection
● Three Microbial Genomics Projects
○ Genomic Purity
○ SNP Method Evaluation
○ Genomic Reference Materials
Talk Overview
Disclaimer
Opinions expressed in this paper are the authors’ and do not necessarily reflect the policies and views of DHS, NIST, or
affiliated venues. Certain commercial equipment, instruments, or materials are identified in this paper in order to specify the
experimental procedure adequately. Such identification is not intended to imply recommendations or endorsement by NIST,
nor is it intended to imply that the materials or equipment identified are necessarily the best available for the purpose.
Microbial Genomics @ NIST

3. External RNA Control Consortium
Advancing Genomic and
Biothreat Detection Metrology
Biothreat Detection
Genome In A Bottle Consortium

4. External
RNA
Control
Consortium
Is the apparent difference between
samples biological or an artifact?

5. External RNA Control Consortium
Spike in controls to assess
technical performance

6. External RNA Control Consortium
https://github.com/usnistgov/erccdashboard
ERCC Dashboard facilitates RM use

7. Is this genetic mutation
real or an artifact?
Genome In A Bottle Consortium

8. http://genomeinabottle.org/
Genome In A Bottle
RM to challenge measurement process

9. http://www.bioplanet.com/gcat
Genome In A Bottle
High confidence variants used
for algorithm benchmarking

10. Biothreat Detection
Is this suspicious powder a biothreat
agent, or a false alarm?

11. Biothreat Detection
Surrogate material to
support first responder training

12. Biothreat Detection
Engineered yeast as surrogate
for biothreat agents

13. Address measurement challenges
with reference materials and
documentary standards.
Summary

14. Microbial Genomics
Microbial Sample Characterization
● Genomic Purity - DHS
● Evaluating SNP calling methods - DHS
● Microbial Genomic RMs - FDA

15. Microbial Genomics Purity
Challenge: Identify low levels of
contaminants without knowing their identity.

16. Genomic Purity
Approach:
Taxonomic read classification
paired with NGS

17. Experimental Design
Genomic Purity
● Seven Organisms
○ Bacillus anthracis
○ Escherichia coli O157:H7
○ Francisella tularensis
○ Pseudomonas aeruginosa
○ Salmonella enterica
○ Staphylococcus aureus
○ Yersinia pestis
● Simulated Datasets
○ Illumina error profile
○ 250 paired end reads
○ 20 X coverage
● 336 spiked datasets
○ Pairwise combinations
○ Contaminant concentrations
5% to 2.5 x 10-4
% of cells
● Pathoscope used for read
classification (http://sourceforge.
net/projects/pathoscope/)
○ Database - Genbank bacterial
genomes
In-Silico Experiment

18. Contaminant
Genomic Purity
Only Contaminant and Sample Genus Identified

19. Genomic Purity
Only Contaminant and Sample Genus Identified

20. Detected contaminants down to 5.0 x 10-4
% of cells
● Sensitivity Dependent on
○ Relative size of the sample and
contaminant genomes
○ Genetic similarity of sample and
contaminant to other organisms in the
database
Genomic Purity

21. Able to Detect Contaminants at less than 1% Cell
Concentrations for Most Pairwise Comparisons
Genomic Purity

22. Conclusions
● Next generation sequencing in conjunction with read classification
algorithms can be used to assess sample purity
● Achieved
○ Genus level classification specificity
○ With sensitivity ranging from 5% to 2.5 x 10-4
%
● Future work includes further validation of the method using real
mixtures
Genomic Purity

23. SNP Method Evaluation
Challenge: Defining confidence in
sample identification

24. SNP Method Evaluation
Approach: Whole genome (SNP)
sample identification

25. SNP Method Evaluation Requirements
1. Reference with known truth
○ Genomic DNA
○ Data
■ real vs. simulated
2. Performance metrics
3. Replicates for assessing
uncertainty
○ multiple sequencing runs
○ bootstrap replicates
○ multiple reference genomes
SNP Method Evaluation

26. Truth Tables can be static or dynamic
SNP Method Evaluation

27. Truth Table Values Used to Calculate Performance Metrics
SNP Method Evaluation
Quality Score Algorithm

28. Conclusions
● Three requirements for evaluation
○ Reference with known truth
○ Performance metrics
○ Replicates for uncertainty assessment
● Working to develop tools for implementing these requirements
● Application of these requirements will help to establish confidence
in SNP based sample identification
SNP Method Evaluation

29. Genomic Reference Materials
Development of Microbial Reference Materials

30. Strains selected based on
public health relevance and GC content
Genomic Reference Materials

31. Orthogonal Methods used to Characterize Genome
Structure, Sequence, and Purity
Genomic Reference Materials

32. ● Microbial genomic reference materials characterized
○ Genome Structure
○ Sequence
○ Purity
○ Stability
● Material and data will help validate pathogen detection
assays as well as sequencing and bioinformatic
workflows.
Genomic Reference Materials
Conclusions

33. Developing a measurement
infrastructure to support genome-
based characterization of microbial
samples.
Microbial Genomics @ NIST
Summary

34. ● Biosystems and Biomaterials Division
● Genome Scale Measurements
Genomic Purity
● Justin Zook
● Nancy Lin
MIcrobial Genomic Reference Materials
● Marc Salit
● Justin Zook
● Steven Lund
● Scott Jackson
● Marc Allard and others at FDA
● Heike Sichtig at the FDA
SNP Calling Method Evaluation
● NIST
○ Jayne Morrow
○ Justin Zook
○ Steven Lund
○ Nancy Lin
○ Marc Salit
● Northern Arizona University
○ Jim Schrup
○ Becky Coleman
○ Jason Sahl
○ Paul Keim
● University of New Hampshire
○ Jeff Foster
This work was supported by the Department of Homeland Security (DHS) Science and Technology
Directorate under the Interagency Agreement HSHQPM-12-X-00078 with NIST and by two interagency
agreements with the FDA.
Acknowledgements