GenomeInABottle, profile picture
Uploaded byGenomeInABottle
PPTX, PDF1,044 views

Giab jan2016 intro and update 160128

The Genome in a Bottle Consortium is developing reference materials, reference methods, and reference data to assess confidence in human whole genome variant calls. The Consortium is characterizing several human genomes including the NA12878 genome, an Ashkenazi Jewish trio, and a Chinese trio from the Personal Genome Project. Data generated for these genomes includes various sequencing technologies from Illumina, Complete Genomics, PacBio, BioNano, and others. The Consortium is developing high-confidence variant calls for SNPs, indels, structural variants, and phasing. Individual datasets and integrated variant calls will be made publicly available on the GIAB FTP site.

Embed presentation

Downloaded 42 times
genomeinabottle.org Genome in a Bottle Consortium January 2016 Stanford University, Stanford, CA Reference Materials for Human Genome Sequencing Marc Salit, Ph.D. and Justin Zook, Ph.D National Institute of Standards and Technology
genomeinabottle.org PGP Trio data described in…
genomeinabottle.org PGP Trio data described in…
genomeinabottle.org August Workshop Description Cell Systems , Volume 1 , Issue 3 , 176 - 177
genomeinabottle.org GIAB Scope • The Genome in a Bottle Consortium is developing the reference materials, reference methods, and reference data needed to assess confidence in human whole genome variant calls. • Priority is authoritative characterization of human genomes. GIAB steering committee, Aug 2015
genomeinabottle.org Genome in a Bottle Consortium Development • NIST met with sequencing technology developers to assess standards needs – Stanford, June 2011 • Open, exploratory workshop – ASHG, Montreal, Canada – October 2011 • Small workshop at NIST to develop consortium for human genome reference materials – FDA, NCBI, NHGRI, NCI, CDC, Wash U, Broad, technology developers, clinical labs, CAP, PGP, Partners, ABRF, others – developed draft work plan – April 2012 • Open, public meetings of GIAB – August 2012 at NIST – March 2013 at Xgen – August 2013 at NIST – January 2014 at Stanford – August 2014 at NIST – January 2015 at Stanford – August 2015 at NIST – January 28-29, 2016 at Stanford – • Website – www.genomeinabottle.org
genomeinabottle.org Well-characterized, stable RMs • Obtain metrics for validation, QC, QA, PT • Determine sources and types of bias/error • Learn to resolve difficult structural variants • Improve reference genome assembly • Optimization • Enable regulated applications Comparison of SNP Calls for NA12878 on 2 platforms, 3 analysis methods
Analytical Performance Sample gDNA isolation Library Prep Sequencing Alignment/Mapping Variant Calling Confidence Estimates Downstream Analysis • Use gDNA reference materials to benchmark performance • Characterized Pilot Genome NA12878 • Ashkenazim Trio, Asian Trio from PGP in process • Tools to facilitate their use – With the Global Alliance Data Working Group Benchmarking Team genericmeasurementprocess
genomeinabottle.org High-confidence SNP/indel calls Zook et al., Nature Biotechnology, 2014. • methods to develop SNP/indel call set described in manuscript • broad and quick adoption of call set for benchmarking – struck nerve
genomeinabottle.org NIST Released the GIAB Pilot Genome as RM 8398 in May 2015 >150 units sold so far
genomeinabottle.org NIST Human Genome Reference Materials (RMs) • NIST RM 8398 is available! – tinyurl.com/giabpilot – DNA isolated from large growth cell cultures – Stable, homogeneous – Best for regulated uses – DNA from same cell line at Coriell (NA12878) • New AJ and Asian Samples – Available from Coriell now – NIST RM available in 2016
genomeinabottle.org Jan 2016 Workshop Thursday • Update and Roadmap • Breakouts – Analyses for PGP GIAB Trios – Reference Material Selection and Development • Breakout reports • Roadmap discussion Friday • Using GIAB Products for technology development, optimization, and demonstration – Experiences from the consortium • Steering committee
genomeinabottle.org Steering Committee Meeting Topics • Future workshops • Format • Program committee? • Crafting a mission statement • Defining scope • Liaison with other efforts Current members – Marc Salit – Justin Zook – David Mittelman – Andrew Grupe – Michael Eberle – Steve Sherry – Deanna Church – Francisco De La Vega – Christian Olsen – Monica Basehore – Lisa Kalman – Christopher Mason – Elizabeth Mansfield – Liz Kerrigan – Leming Shi – Melvin Limson – Alexander Wait Zaranek – Nils Homer – Fiona Hyland – Steve Lincoln – Don Baldwin – Robyn Temple-Smolkin – Chunlin Xiao – Kara Norman – Luke Hickey
genomeinabottle.org Agenda Monday • Breakfast and registration • Welcome and Context Setting • NIST RM Update and Status Report • Charge to Working Groups • Coffee Break • Working Group Breakout Discussions • Lunch (provided) • Informal Working Group Reports • Coffee Break • Breakout Topical Discussions – Topic #1: Moving beyond the 'easy' variants and regions of the genome – Topic #2: Selecting future genomes for Reference Materials Tuesday • Breakfast and registration • Use cases: Experiences using the pilot Reference Material • Discussion of plans to release pilot Reference Material • Coffee Break • Working Group Breakout discussions • Lunch (provided) • Working Group leaders present plans and discussion • Steering committee Overview • First meeting of the Steering Committee (others adjourn) Please Note Slides will be made available on SlideShare after the workshop (see genomeinabottle.org). Tweets are welcome unless the speaker requests otherwise. Please use #giab as the hashtag.
We are liaising with… • Illumina Platinum Genomes • CDC GeT-RM • Korean Genome Project • Genome Reference Consortium • 1000 Genomes SV group • CAP/CLIA • ABRF • FDA • SEQC • Global metrology system • Global Alliance for Genomics and Health Benchmarking Team • NCBI/CDC GeT-RM Browser • GCAT website
NGS RM Project Slide courtesy of Lisa Kalman, CDC
Association of Biomolecular Resource Facilities (ABRF) www.abrf.org Next Generation Sequencing Study Phase 2: DNA sequencing platforms Study Design and Launch Plan Slides courtesy of Chris Mason January 24, 2016
Aims Create reference data sets - Sequence data from reference samples will be generated with intra- and inter-lab replication to model the likely range of performance that should be expected under normal service laboratory conditions. Test and create reference samples - Designated reference samples will be easily accessible to the community for self-evaluation by comparison to the reference data. Samples should be standardized, able to be stably reproduced over time, and suitable for development of new laboratory and bioinformatics methods. Data release and Immediate utility - Performance metrics and data will be developed for instrument platforms and sample preparation protocols that are deployed now or will be in the near future in core sequencing facilities. After QC, data will be released to the entire Genome in a Bottle (GIAB) and ABRF Consortia for use and preparation for submission as publications. ABRF NGS Phase II Study
Samples and Platforms – All tested in triplicate across three distinct sites Platform Human DNA Bacterial DNA Illumina HiSeq X Ten A, B, C, C2, C2f Illumina HiSeq 4000 A, B, C Illumina HiSeq 2500 v4 1T A, B, C Illumina HiSeq 2500 v3 Rapid Run C Ste, Eco, Mil, P Illumina NextSeq 500 High Output C Illumina MiSeq Ste, Eco, Mil, P Life Tech Proton A, B, C exomes Ste, Eco, Mil, P Life Tech S5 A, B, C exomes Ste, Eco, Mil, P Life Tech PGM Ste, Eco, Mil, P Pacific Biosciences Ste, Eco, Mil, P Oxford Nanopore Ste, Eco, Mil, P maternal paternal son son (Coriell) A B C C2 Ste Eco Mil pool Human Trio Bacterial Isolates and Mixture ABRF NGS Phase II Study
Reference DNA, TruSeq PCR-free 350 FFPE DNA, TruSeq Nano FFPE DNA, TruSeq PCR-free KAPA libraries from sites a-b-c Ste Eco Mil pool maternal paternal son son (Coriell) %GC: 28 50 72 A B C C2 Personal Genome Project NIST Reference Human Genomes C2f Reference bacterial genomes TruSeq PCR-free 550 Ca Illumina (ILMN) - Samples ABRF NGS Phase II Study
Organization and Leadership ABRF NGS Phase II Study
Sequencing Quality Control Phase II (SEQC2) – An Introduction Slides courtesy of Weida Tong, Ph.D. Division of Bioinformatics and Biostatistics, NCTR/FDA 22
Short reads vs long reads Detection power for rare mutation Detection accuracy for difficult genes Application scope of MiSeq Variants call (e.g., SNV, CNV, Indels) Assess the WGS accuracy and reproducibility for variants call by investigating the join effect of reads alignment pipelines, variants call methods and coverage as well as comparing the results from personal genome versus reference genome. Assess detection power of ultra-deep sequencing (TGS) for subclonal mutation and its dependency on bioinformatics and coverage. Assess the utility of MiSeq for (1) detection of subclonal mutation, (2) the difficult genes (e.g., HLA), and (3) the difficult variations (e.g., Indels) Assess the accuracy for some difficult genes that varies significantly due to complexity in their genomic regions (e.g. GC region) with specifically focused on HLA genes. Datasets: • Approaches: WGS and TGS • Platforms: Hiseq, PacBio, MiSeq, etc • Samples: TRIOs, NB, cell lines, etc Parameters: • Personal vs reference genome • Bioinformatics • Coverage SEQC2 Overview Assess short reads alone, long reads alone and their combination for genome assembly and subsequent variant calling in WGS.
Trio Study Coverage/platform Notes Short reads Long reads SEQC2: HapMap Trio (European) 80x TBD Planned for both WGS and TGS; genotyping data and information from HapMap are available GIAB: Trio (Ashkenazim) Illumina 300x 69x (son), 30x (parents) This dataset is generated by Genome In A Bottle (GIAB) consortium. We closely work with GIAB to obtain the update information of this Trio and the GIAB leaders also participate in SEQC2. Complete Genomics BioNano Ion Torrent Moleculo SOLiD (WGS) SEQC2: Chinese Trio and test of LCL- germline 100x 50x Panned; the datasets will be provided by Dr. Leming Shi who is a part of SEQC2 leadership team. Three Trio Datasets 24
Candidate NIST Reference Materials Genome PGP ID Coriell ID NIST ID NIST RM # CEPH Mother/Daugh ter N/A GM12878 HG001 RM8398 AJ Son huAA53E0 GM24385 HG002 RM8391 (son)/RM8392 (trio) AJ Father hu6E4515 GM24149 HG003 RM8392 (trio) AJ Mother hu8E87A9 GM24143 HG004 RM8392 (trio) Asian Son hu91BD69 GM24631 HG005 RM8393 Asian Father huCA017E GM24694 N/A N/A Asian Mother hu38168C GM24695 N/A N/A
NIST Microbial Genomic DNA Reference Materials Credit: Nate Olson
Analysis process for Microbial RMs Credit: Nate Olson
genomeinabottle.org GIAB Progress Update January 2016
Dataset AJ Son AJ Parents Chinese son Chinese parents NA12878 Illumina Paired- end X X X X X Illumina Long Mate pair X X X X X Illumina “moleculo” X X X X X Complete Genomics X X X X X Complete Genomics LFR X X X Ion exome X X X X BioNano X X X X 10X X X X PacBio X X X SOLiD single end X X X Illumina exome X X X X Oxford Nanopore X
0 200 400 600 800 1000 1200 1400 1600 1800 2000 0 10000 20000 30000 40000 50000 60000 70000 80000 90000 #IPs Month GIAB ftp site downloads/unique-IPs by month
GIAB Analysis Group – New Data Sets Leaders • Francisco de la Vega – Stanford, TOMA Biosciences • Chris Mason – Weil Cornell Medical Center • Tina Graves – Washington University • Valerie Schneider – NCBI •and Justin and Marc Strategic Documents • Analysis Group Responsibilities: – https://docs.google.com/document/d/10e A0DwB4iYTSFM_LPO9_2LyyN2xEqH49OXH htNH1uzw/edit?usp=sharing • Analysis Milestones: – https://docs.google.com/spreadsheets/d/1Pj4nSz H742g40wJz2fA6f8kFtZYAToZpSZYVPiC5st4/edit?u sp=sharing • Analysis Methods – https://docs.google.com/spreadsheet s/d/1Je2g85H7oK6kMXbBOoqQ1FM NrvGnFuUJTJn7deyYiS8/edit?usp=sha ring • Analysis Plan: – https://drive.google.com/file/d/0B7Ao1qq JJDHQdnVEaVdqbWdEdkE/view?usp=shari ng • Collecting Data and analyses on GIAB FTP Site • Recruiting people to help with the work. Goal: Establish and distribute a set of authoritative benchmark variant calls of all types and sizes, as well as homozygous reference regions, on GIAB PGP trios
GIAB Analysis Group – New Data Sets Types of analyses • SNPs/indels – NIST working on integration – 10X/moleculo/PacBio for difficult-to-map regions • Assembly – 2 de novo assemblies – Being used for SV calling Status • Structural variants – Candidate calls being generated by 15+ groups with >20 different algorithms and 6 datasets – 3+ integration methods – ~monthly calls • Long-range Phasing – 2 phased calls so far (CG LFR and 10X) – Integration methods needed • Methylation analyses Goal: Establish and distribute a set of authoritative benchmark variant calls of all types and sizes, as well as homozygous reference regions, on GIAB PGP trios
genomeinabottle.org SNP/Indel Integration Method Update • Implementing refined integration methods – Developed so others can readily reproduce results – Consistent results for all GIAB genomes – Simpler process taking advantage of best practices for each technology • Validating with released NA12878 RM data – Preliminary comparisons show minor changes • Application to PGP trios – Plan to analyze AJ trio by Q2 2016 – Release of NIST RMs in Q2 2016 – Develop calls for GRCh38
genomeinabottle.org Data Release: Real-time, Open, Public Release Individual Datasets • Uploaded to GIAB FTP site as it is collected • Includes raw reads, aligned reads, and variant/reference calls Integrated High-confidence Calls • First develop SNP, indel, and homozygous reference calls • Then develop SV and non- SV calls • Released calls are versioned • Preliminary callsets will be made available to be critiqued
GIAB AJ Trio Hybrid PacBio/BioNano Assembly Hybrid (PacBio with BioNano) Input Assembly Notes # of Scaffolds N50 Max Total HG002 Falcon 248 22.7Mb 92.8Mb 2.38Gb Trio Falcon 210 29.3Mb 87.6Mb 2.32Gb Two Step Trio celera (child) + falcon (trio) 187 34.3Mb 98.0Mb 2.6Gb Credits: Ali Bashir, Jason Chin, Alex Hastie Pendleton et al, Nature Methods, 2015
svclassify
Proposed approach to form high- confidence SV (and non-SV) calls Generate Candidate Calls Compare/evaluate calls using Parliament/MetaSV/svclassify/others?; manual inspection Integrate new and revised calls; manual inspection Combine integrated calls; manual inspection; targeted experimental validation? August 30, 2015 January 2016 Plan in January 2016 Feb 2016 and beyond
Deletion overlap summary for son By # of callsets # of callsets # of calls 1 3780 2 1391 3 859 4 574 5+ 344 By Technology Technology # of calls Illumina 3277 PacBio 5177 BioNano 812 CG 1758 Illumina/CG+PacBio 2318 Illumina/CG+BioNano 518 PacBio+BioNano 467 2+ technologies 2661 Converted all to bed; combined with bedtools multiinter; Calls within 50bps were merged
Preliminary Confirmation of SVs Integration results from AJ son Parliament: BMC Genomics, 2015, 16:286 (performed by Andrew Carroll, DNAnexus) MetaSV: Bioinformatics, 2015, 31:2741 (performed by Marghoob Mohiyuddin, Bina/Roche) • Parliament – Candidates from Illumina – Confirmed by PacBio and/or Illumina – ~50% in both technologies – ~4.5k deletions, 1k insertions – 85% of Genotypes consistent within Trio • MetaSV – Multiple types of evidence from Illumina MetaSV Total: 2809 Parliament Total: 5467 569 (20 %) 977 (18 %) MetaSV 2240 (80 %) Parliament 4490 (82 %) 50 % reciprocal overlap Some overlap within Parliament calls
genomeinabottle.org GeT-RM Browser from NCBI and CDC • http://www.ncbi.nlm.nih.gov/variation/tools/get-rm/ • Allows visualization of data underlying call each call
genomeinabottle.org Uses of GIAB NA12878 Oncology – Molecular and Cellular Tumor Markers “Next Generation” Sequencing (NGS) guidelines for somatic genetic variant detection www.bioplanet.com/gcat
Global Alliance for Genomics and Health Benchmarking Task Team • Initial version of standardized definitions for performance metrics like TP, FP, and FN. • Continued development of sophisticated benchmarking tools • vcfeval – Len Trigg • hap.py – Peter Krusche • vgraph – Kevin Jacobs • Standardized intermediate and final file formats • Standardized bed files with difficult genome contexts for stratification • Simulating reads with non-SNP ClinVar variants to demonstrate importance of these tools • github.com/ga4gh/benchmarking -tools Next steps • Further analysis to demonstrate importance of sophisticated tools • Write manuscript about the team’s tools • Integrate vcfeval and hap.py to take advantage of strengths of each • Recommend “Best Practices” for benchmarking • Explore venues for making the team’s benchmarking process easier to use Progress
Proposed Performance Metrics Definitions • Define TP/FP/FN/TN in 4 ways depending on required stringency of match: • Loose match: TP if within x-bp of a true variant • Allelle match: TP if ALT allele matches • Genotype match: TP if genotype and ALT allele match • Phasing match: TP if genotype, ALT allele, and phasing with nearby variants all match • True negatives are difficult to define because an infinite number of potential alleles exist
genomeinabottle.org Global Alliance for Genomics and Health Benchmarking Task Team Credit: Rebecca Truty, Complete Genomics How should we interpret this complex variant on chr21?
GA4GH Benchmarking Tool Architecture Truth VCF Query VCF Comparison Engine vcfeval / vgraph / xcmp / bcftools / ... VCF-I Two-column VCF with TP/FP/FN annotations Quantification e.g. quantify / hap.py Stratification BED files Confident Call Regions VCF-R Two-column VCF with TP/FP/FN/UNK annotations Counts Credit: Peter Krusche https://github.com/ga4gh/benchmarking-tools
Approaches to Benchmarking Variant Calling • Well-characterized whole genome Reference Materials • Many samples characterized in clinically relevant regions • Synthetic DNA spike-ins • Cell lines with engineered mutations • Simulated reads • Modified real reads • Modified reference genomes • Confirming results found in real samples over time
Challenges in Benchmarking Small Variant Calling • It is difficult to do robust benchmarking of tests designed to detect many analytes (e.g., many variants) • Easiest to benchmark only within high-confidence bed file, but… • Benchmark calls/regions tend to be biased towards easier variants and regions – Some clinical tests are enriched for difficult sites • Challenges with benchmarking complex variants near boundaries of high-confidence regions • Always manually inspect a subset of FPs/FNs • Stratification by variant type and region is important • Always calculate confidence intervals on performance metrics
Particular Challenges in Benchmarking SV Calling • How to establish benchmark calls for difficult regions? • How to establish non-SV regions to assess FP rates? • Multiple dimensions of accuracy: – Predicted SV existence – Predicted SV type – Predicted size – Predicted breakpoints – Predicted exact sequence
Acknowledgments • FDA – Elizabeth Mansfield • Many members of Genome in a Bottle – New members welcome! – Sign up on website for email newsletters GIAB Steering Committee – Marc Salit – Justin Zook – David Mittelman – Andrew Grupe – Michael Eberle – Steve Sherry – Deanna Church – Francisco De La Vega – Christian Olsen – Monica Basehore – Lisa Kalman – Christopher Mason – Elizabeth Mansfield – Liz Kerrigan – Leming Shi – Melvin Limson – Alexander Wait Zaranek – Nils Homer – Fiona Hyland – Steve Lincoln – Don Baldwin – Robyn Temple-Smolkin – Chunlin Xiao – Kara Norman – Luke Hickey
For More Information www.genomeinabottle.org - sign up for general GIAB and Analysis Team google group emails github.com/genome-in-a-bottle – Guide to GIAB data & ftp www.slideshare.net/genomeinabottle www.ncbi.nlm.nih.gov/variation/tools/get-rm/ - Get-RM Browser Data: http://biorxiv.org/content/early/2015/09/15/026468 Global Alliance Benchmarking Team – https://github.com/ga4gh/benchmarking-tools Twice yearly public workshops – Winter at Stanford University, California, USA – Summer at NIST, Maryland, USA Justin Zook: jzook@nist.gov Marc Salit: salit@nist.gov
GIAB Roadmap: Where are we, Where are we going? • Reference Materials – Germline – Somatic • Informatics – Analysis of GIAB data – Benchmarking • Documentary Standards/Publications – Documentation of methods – Supporting Use

More Related Content

PPTX
ASHG 2015 Genome in a bottle
byGenomeInABottle
 
PPTX
Jan2016 bina giab
byGenomeInABottle
 
PPTX
Giab ashg webinar 160224
byGenomeInABottle
 
PPTX
Aug2015 Ali Bashir and Jason Chin Pac bio giab_assembly_summary_ali3
byGenomeInABottle
 
PPTX
Aug2015 salit standards architecture
byGenomeInABottle
 
PPTX
Giab jan2016 analysis team breakout summary
byGenomeInABottle
 
PPTX
Genome in a bottle april 30 2015 hvp Leiden
byGenomeInABottle
 
PPTX
160627 giab for festival sv workshop
byGenomeInABottle
 
ASHG 2015 Genome in a bottle
byGenomeInABottle
 
Jan2016 bina giab
byGenomeInABottle
 
Giab ashg webinar 160224
byGenomeInABottle
 
Aug2015 Ali Bashir and Jason Chin Pac bio giab_assembly_summary_ali3
byGenomeInABottle
 
Aug2015 salit standards architecture
byGenomeInABottle
 
Giab jan2016 analysis team breakout summary
byGenomeInABottle
 
Genome in a bottle april 30 2015 hvp Leiden
byGenomeInABottle
 
160627 giab for festival sv workshop
byGenomeInABottle
 

What's hot

PPTX
GIAB-GRC workshop oct2015 giab introduction 151005
byGenomeInABottle
 
PPTX
Giab jan2016 analysis team breakout SNP indel update zook
byGenomeInABottle
 
PPTX
Giab aug2015 intro and update 150821.pptx
byGenomeInABottle
 
PPTX
171114 best practices for benchmarking variant calls justin
byGenomeInABottle
 
PPTX
160628 giab for festival of genomics
byGenomeInABottle
 
PPTX
Giab workshop intro 180125
byGenomeInABottle
 
PPTX
Aug2015 Giab nist integration methods
byGenomeInABottle
 
PPTX
Genome in a Bottle
byGenome Reference Consortium
 
PPTX
Jan2016 horizon GIAB
byGenomeInABottle
 
PPTX
161115 precision fda giab
byGenomeInABottle
 
PPTX
GIAB Integrating multiple technologies to form benchmark SVs 180517
byGenomeInABottle
 
PPTX
Aug2015 horizon diagnostics
byGenomeInABottle
 
PPTX
170120 giab stanford genetics seminar
byGenomeInABottle
 
PPTX
Tools for Using NIST Reference Materials
byGenomeInABottle
 
PPTX
GIAB Sep2016 Lightning megan cleveland targeted seq
byGenomeInABottle
 
PPTX
Jan2016 rm selection and design breakout summary
byGenomeInABottle
 
PDF
2016 ashg giab poster
byGenomeInABottle
 
PPTX
Aug2013 illumina platinum genomes
byGenomeInABottle
 
PPTX
Giab for jax long read 190917
byGenomeInABottle
 
PPTX
Sept2016 plenary nist_intro
byGenomeInABottle
 
GIAB-GRC workshop oct2015 giab introduction 151005
byGenomeInABottle
 
Giab jan2016 analysis team breakout SNP indel update zook
byGenomeInABottle
 
Giab aug2015 intro and update 150821.pptx
byGenomeInABottle
 
171114 best practices for benchmarking variant calls justin
byGenomeInABottle
 
160628 giab for festival of genomics
byGenomeInABottle
 
Giab workshop intro 180125
byGenomeInABottle
 
Aug2015 Giab nist integration methods
byGenomeInABottle
 
Genome in a Bottle
byGenome Reference Consortium
 
Jan2016 horizon GIAB
byGenomeInABottle
 
161115 precision fda giab
byGenomeInABottle
 
GIAB Integrating multiple technologies to form benchmark SVs 180517
byGenomeInABottle
 
Aug2015 horizon diagnostics
byGenomeInABottle
 
170120 giab stanford genetics seminar
byGenomeInABottle
 
Tools for Using NIST Reference Materials
byGenomeInABottle
 
GIAB Sep2016 Lightning megan cleveland targeted seq
byGenomeInABottle
 
Jan2016 rm selection and design breakout summary
byGenomeInABottle
 
2016 ashg giab poster
byGenomeInABottle
 
Aug2013 illumina platinum genomes
byGenomeInABottle
 
Giab for jax long read 190917
byGenomeInABottle
 
Sept2016 plenary nist_intro
byGenomeInABottle
 

Viewers also liked

PPTX
Jan2016 dnanexus giab uses andrew carroll
byGenomeInABottle
 
PDF
Jan2016 curoverse benchmarking somatic variant_calling_pipelines
byGenomeInABottle
 
PDF
Successful detection of 40 COSMIC hotspot mutations at allelic frequency belo...
byThermo Fisher Scientific
 
PPTX
150219 agbt giab_poster_marc
byGenomeInABottle
 
PPT
A National Network of Biomedical Research Expertise
byManinder Kahlon
 
PPTX
Leadership in Decline: Assessing U.S. International Competitiveness in Biomed...
byInformation Technology and Innovation Foundation
 
PDF
Standards and tools for model management in biomedical research
byUniversity Medicine Greifswald
 
PPTX
Maximizing Social Capital to Increase Core Facility Exposure and Usage
byRyan Duggan
 
PDF
Genome in a Bottle Consortium Workshop Welcome Aug. 16
byGenomeInABottle
 
PPTX
Biomedical research
byAnjo Yumol
 
PPT
I V I F2 F July 2005 Talk
bybattagline
 
PPTX
Information Sciences Solutions to Core Facility Problems at St. Jude Children...
byMatt Stine
 
KEY
Case Study: SRM 2.0 - A next generation shared resource management system bui...
byMatt Stine
 
PDF
George Church: Standards & Open-Access Genome-Environment-Trait Data
byGenomeInABottle
 
PDF
NIST program to develop genomic reference materials
byGenomeInABottle
 
PPTX
Clean Labs Training
byDalibor Ĺ ulc
 
PDF
decentralization: a trend in biomedical research
byBrian Bot
 
PPTX
170326 giab abrf
byGenomeInABottle
 
PPTX
Making Biomedical Research More Like Airbnb
byPhilip Bourne
 
PPT
Cross-Disciplinary Biomedical Research at Calit2
byLarry Smarr
 
Jan2016 dnanexus giab uses andrew carroll
byGenomeInABottle
 
Jan2016 curoverse benchmarking somatic variant_calling_pipelines
byGenomeInABottle
 
Successful detection of 40 COSMIC hotspot mutations at allelic frequency belo...
byThermo Fisher Scientific
 
150219 agbt giab_poster_marc
byGenomeInABottle
 
A National Network of Biomedical Research Expertise
byManinder Kahlon
 
Leadership in Decline: Assessing U.S. International Competitiveness in Biomed...
byInformation Technology and Innovation Foundation
 
Standards and tools for model management in biomedical research
byUniversity Medicine Greifswald
 
Maximizing Social Capital to Increase Core Facility Exposure and Usage
byRyan Duggan
 
Genome in a Bottle Consortium Workshop Welcome Aug. 16
byGenomeInABottle
 
Biomedical research
byAnjo Yumol
 
I V I F2 F July 2005 Talk
bybattagline
 
Information Sciences Solutions to Core Facility Problems at St. Jude Children...
byMatt Stine
 
Case Study: SRM 2.0 - A next generation shared resource management system bui...
byMatt Stine
 
George Church: Standards & Open-Access Genome-Environment-Trait Data
byGenomeInABottle
 
NIST program to develop genomic reference materials
byGenomeInABottle
 
Clean Labs Training
byDalibor Ĺ ulc
 
decentralization: a trend in biomedical research
byBrian Bot
 
170326 giab abrf
byGenomeInABottle
 
Making Biomedical Research More Like Airbnb
byPhilip Bourne
 
Cross-Disciplinary Biomedical Research at Calit2
byLarry Smarr
 

Similar to Giab jan2016 intro and update 160128

PPTX
Aug2014 giab intro slides
byGenomeInABottle
 
PPTX
Jan2015 GIAB intro, Update, and Data Analysis Planning
byGenomeInABottle
 
PPTX
140127 GIAB Intro
byGenomeInABottle
 
PPTX
Aug2013 NIST program slides
byGenomeInABottle
 
PPTX
Genome in a Bottle - Towards new benchmarks for the “dark matter” of the huma...
byGenomeInABottle
 
PPTX
Using accurate long reads to improve Genome in a Bottle Benchmarks 220923
byGenomeInABottle
 
PPTX
GIAB Benchmarks for SVs and Repeats for stanford genetics sv 200511
byGenomeInABottle
 
PPTX
Aug2013 reference material selection and design working group
byGenomeInABottle
 
PPTX
GIAB update for GRC GIAB workshop 191015
byGenomeInABottle
 
PPTX
150224 giab 30 min generic slides
byGenomeInABottle
 
PPTX
171017 giab for giab grc workshop
byGenome Reference Consortium
 
PPTX
140127 GIAB update and NIST high-confidence calls
byGenomeInABottle
 
PPTX
Genome in a bottle for next gen dx v2 180821
byGenomeInABottle
 
PPTX
2014 agbt giab_progress update
byGenomeInABottle
 
PPTX
GIAB for AMP GeT-RM Forum
byGenomeInABottle
 
PDF
Giab agbt small_var_2020
byGenomeInABottle
 
PPTX
171017 giab for giab grc workshop
byGenomeInABottle
 
PPTX
Genome in a bottle for amp GeT-RM 181030
byGenomeInABottle
 
PPTX
Genome in a bottle for ashg grc giab workshop 181016
byGenomeInABottle
 
PDF
2017 agbt giab_poster
byGenomeInABottle
 
Aug2014 giab intro slides
byGenomeInABottle
 
Jan2015 GIAB intro, Update, and Data Analysis Planning
byGenomeInABottle
 
140127 GIAB Intro
byGenomeInABottle
 
Aug2013 NIST program slides
byGenomeInABottle
 
Genome in a Bottle - Towards new benchmarks for the “dark matter” of the huma...
byGenomeInABottle
 
Using accurate long reads to improve Genome in a Bottle Benchmarks 220923
byGenomeInABottle
 
GIAB Benchmarks for SVs and Repeats for stanford genetics sv 200511
byGenomeInABottle
 
Aug2013 reference material selection and design working group
byGenomeInABottle
 
GIAB update for GRC GIAB workshop 191015
byGenomeInABottle
 
150224 giab 30 min generic slides
byGenomeInABottle
 
171017 giab for giab grc workshop
byGenome Reference Consortium
 
140127 GIAB update and NIST high-confidence calls
byGenomeInABottle
 
Genome in a bottle for next gen dx v2 180821
byGenomeInABottle
 
2014 agbt giab_progress update
byGenomeInABottle
 
GIAB for AMP GeT-RM Forum
byGenomeInABottle
 
Giab agbt small_var_2020
byGenomeInABottle
 
171017 giab for giab grc workshop
byGenomeInABottle
 
Genome in a bottle for amp GeT-RM 181030
byGenomeInABottle
 
Genome in a bottle for ashg grc giab workshop 181016
byGenomeInABottle
 
2017 agbt giab_poster
byGenomeInABottle
 

More from GenomeInABottle

PDF
2023 GIAB AMP Update
byGenomeInABottle
 
PDF
GIAB Tumor Normal ASHG 2023
byGenomeInABottle
 
PDF
Stratomod ASHG 2023
byGenomeInABottle
 
PDF
GIAB_ASHG_JZook_2023.pdf
byGenomeInABottle
 
PPTX
Benchmarking with GIAB 220907
byGenomeInABottle
 
PPTX
Genome in a Bottle- reference materials to benchmark challenging variants and...
byGenomeInABottle
 
PPTX
GIAB Technical Germline Benchmark roadmap discussion
byGenomeInABottle
 
PDF
Ga4gh 2019 - Assuring data quality with benchmarking tools from GIAB and GA4GH
byGenomeInABottle
 
PPTX
GIAB ASHG 2019 Structural Variant poster
byGenomeInABottle
 
PDF
GIAB GRC Workshop ASHG 2019 Billy Rowell Evaluation of v4 with CCS GATK
byGenomeInABottle
 
PPTX
GIAB ASHG 2019 Small Variant poster
byGenomeInABottle
 
PPTX
GRC GIAB Workshop ASHG 2019 Small Variant Benchmark
byGenomeInABottle
 
PPTX
Jason Chin MHC diploid assembly
byGenomeInABottle
 
PPTX
GIAB and long reads for bio it world 190417
byGenomeInABottle
 
PDF
New methods diploid assembly with graphs
byGenomeInABottle
 
PDF
How giab fits in the rest of the world seqc2 tumor normal
byGenomeInABottle
 
PDF
New data from giab genomes pacbio ccs
byGenomeInABottle
 
PPT
New data from giab genomes strand-seq
byGenomeInABottle
 
PDF
New data from giab genomes promethion
byGenomeInABottle
 
PPTX
New data from giab genomes intro and ultralong nanopore
byGenomeInABottle
 
2023 GIAB AMP Update
byGenomeInABottle
 
GIAB Tumor Normal ASHG 2023
byGenomeInABottle
 
Stratomod ASHG 2023
byGenomeInABottle
 
GIAB_ASHG_JZook_2023.pdf
byGenomeInABottle
 
Benchmarking with GIAB 220907
byGenomeInABottle
 
Genome in a Bottle- reference materials to benchmark challenging variants and...
byGenomeInABottle
 
GIAB Technical Germline Benchmark roadmap discussion
byGenomeInABottle
 
Ga4gh 2019 - Assuring data quality with benchmarking tools from GIAB and GA4GH
byGenomeInABottle
 
GIAB ASHG 2019 Structural Variant poster
byGenomeInABottle
 
GIAB GRC Workshop ASHG 2019 Billy Rowell Evaluation of v4 with CCS GATK
byGenomeInABottle
 
GIAB ASHG 2019 Small Variant poster
byGenomeInABottle
 
GRC GIAB Workshop ASHG 2019 Small Variant Benchmark
byGenomeInABottle
 
Jason Chin MHC diploid assembly
byGenomeInABottle
 
GIAB and long reads for bio it world 190417
byGenomeInABottle
 
New methods diploid assembly with graphs
byGenomeInABottle
 
How giab fits in the rest of the world seqc2 tumor normal
byGenomeInABottle
 
New data from giab genomes pacbio ccs
byGenomeInABottle
 
New data from giab genomes strand-seq
byGenomeInABottle
 
New data from giab genomes promethion
byGenomeInABottle
 
New data from giab genomes intro and ultralong nanopore
byGenomeInABottle
 

Recently uploaded

PPTX
Gut Microbial Metabolites as Cancer Immunomodulators
byArchitMehta11
 
PPTX
Oncology Physiotherapy_ A Key Piece in Cancer Care Puzzle.pptx
byasmitashresthapt
 
PPTX
PERCEPTION.pptx I SEMESTER STUDENTS BY SHILPA HOTAKAR
bySHILPA HOTAKAR
 
PPTX
THYROID GLAND ANATOMY PHYSIOLOGY AND DISORDER
byKrantiKhobragade
 
PPTX
2026 ADA Standards of Care:Summary of Changes
byDr. Om J Lakhani
 
PPTX
NERVOUS SYSTEM11 by Gulshan athbhaiya.pptx
byGulshan Athbhaiya
 
PDF
preoperative assessment Pheochromocytoma 2025.pdf
byAssist. Prof. Radhwan Hazem Alkhashab
 
PPTX
Biotransformation Of Cortisone By Using Rhodococcus rhodnii
bySadaqat Ahmed
 
PDF
Pòster "Virtual Reality in Pain Management"
byBadalona Serveis Assistencials
 
PPTX
Innovations in Chronic Pain Management: From Clinical Idea to Published Research
byDaradia: The Pain Clinic
 
PPTX
DEBATE ON LUNG CANCER PACIFIC VS KEYNOTE TRIAL
byKanhu Charan
 
PPTX
Gestational Diabetes: A Journey from Detection to Delivery
byViresh Mahajani
 
PDF
Role of AI in Autism spectrum disorder (ASD)
byShraddhaJha15
 
PDF
Definitive Innovation in Resectable Head and Neck Cancer: New Immunotherapy A...
byPVI, PeerView Institute for Medical Education
 
PPTX
DEC 2025ONCOLOGY CARTOONS BY DR KANHU CHARAN PATRO
byKanhu Charan
 
PPTX
Herbs As Health Food....pptx ( Pharmacognosy )
byBushraDeshpande
 
PDF
Filling in the Gaps in Hemophilia Guideline Recommendations: Evidence-Based S...
byi3 Health
 
PDF
My swollen feet due to heart problem at Allenby 91
byHaim R. Branisteanu
 
PPTX
FAMILY PLANING METHOD AND PROGRAMME IN BRIEF
byKrantiKhobragade
 
PPTX
Emerging Targets in NSCLC beyond EGFR, ALK & ROS .pptx
bySumitKumar452
 
Gut Microbial Metabolites as Cancer Immunomodulators
byArchitMehta11
 
Oncology Physiotherapy_ A Key Piece in Cancer Care Puzzle.pptx
byasmitashresthapt
 
PERCEPTION.pptx I SEMESTER STUDENTS BY SHILPA HOTAKAR
bySHILPA HOTAKAR
 
THYROID GLAND ANATOMY PHYSIOLOGY AND DISORDER
byKrantiKhobragade
 
2026 ADA Standards of Care:Summary of Changes
byDr. Om J Lakhani
 
NERVOUS SYSTEM11 by Gulshan athbhaiya.pptx
byGulshan Athbhaiya
 
preoperative assessment Pheochromocytoma 2025.pdf
byAssist. Prof. Radhwan Hazem Alkhashab
 
Biotransformation Of Cortisone By Using Rhodococcus rhodnii
bySadaqat Ahmed
 
Pòster "Virtual Reality in Pain Management"
byBadalona Serveis Assistencials
 
Innovations in Chronic Pain Management: From Clinical Idea to Published Research
byDaradia: The Pain Clinic
 
DEBATE ON LUNG CANCER PACIFIC VS KEYNOTE TRIAL
byKanhu Charan
 
Gestational Diabetes: A Journey from Detection to Delivery
byViresh Mahajani
 
Role of AI in Autism spectrum disorder (ASD)
byShraddhaJha15
 
Definitive Innovation in Resectable Head and Neck Cancer: New Immunotherapy A...
byPVI, PeerView Institute for Medical Education
 
DEC 2025ONCOLOGY CARTOONS BY DR KANHU CHARAN PATRO
byKanhu Charan
 
Herbs As Health Food....pptx ( Pharmacognosy )
byBushraDeshpande
 
Filling in the Gaps in Hemophilia Guideline Recommendations: Evidence-Based S...
byi3 Health
 
My swollen feet due to heart problem at Allenby 91
byHaim R. Branisteanu
 
FAMILY PLANING METHOD AND PROGRAMME IN BRIEF
byKrantiKhobragade
 
Emerging Targets in NSCLC beyond EGFR, ALK & ROS .pptx
bySumitKumar452
 

Giab jan2016 intro and update 160128

  • 1.
    genomeinabottle.org Genome in aBottle Consortium January 2016 Stanford University, Stanford, CA Reference Materials for Human Genome Sequencing Marc Salit, Ph.D. and Justin Zook, Ph.D National Institute of Standards and Technology
  • 2.
  • 3.
  • 4.
    genomeinabottle.org August Workshop Description CellSystems , Volume 1 , Issue 3 , 176 - 177
  • 5.
    genomeinabottle.org GIAB Scope • TheGenome in a Bottle Consortium is developing the reference materials, reference methods, and reference data needed to assess confidence in human whole genome variant calls. • Priority is authoritative characterization of human genomes. GIAB steering committee, Aug 2015
  • 6.
    genomeinabottle.org Genome in aBottle Consortium Development • NIST met with sequencing technology developers to assess standards needs – Stanford, June 2011 • Open, exploratory workshop – ASHG, Montreal, Canada – October 2011 • Small workshop at NIST to develop consortium for human genome reference materials – FDA, NCBI, NHGRI, NCI, CDC, Wash U, Broad, technology developers, clinical labs, CAP, PGP, Partners, ABRF, others – developed draft work plan – April 2012 • Open, public meetings of GIAB – August 2012 at NIST – March 2013 at Xgen – August 2013 at NIST – January 2014 at Stanford – August 2014 at NIST – January 2015 at Stanford – August 2015 at NIST – January 28-29, 2016 at Stanford – • Website – www.genomeinabottle.org
  • 7.
    genomeinabottle.org Well-characterized, stable RMs •Obtain metrics for validation, QC, QA, PT • Determine sources and types of bias/error • Learn to resolve difficult structural variants • Improve reference genome assembly • Optimization • Enable regulated applications Comparison of SNP Calls for NA12878 on 2 platforms, 3 analysis methods
  • 8.
    Analytical Performance Sample gDNA isolation LibraryPrep Sequencing Alignment/Mapping Variant Calling Confidence Estimates Downstream Analysis • Use gDNA reference materials to benchmark performance • Characterized Pilot Genome NA12878 • Ashkenazim Trio, Asian Trio from PGP in process • Tools to facilitate their use – With the Global Alliance Data Working Group Benchmarking Team genericmeasurementprocess
  • 9.
    genomeinabottle.org High-confidence SNP/indel calls Zooket al., Nature Biotechnology, 2014. • methods to develop SNP/indel call set described in manuscript • broad and quick adoption of call set for benchmarking – struck nerve
  • 10.
    genomeinabottle.org NIST Released theGIAB Pilot Genome as RM 8398 in May 2015 >150 units sold so far
  • 11.
    genomeinabottle.org NIST Human Genome ReferenceMaterials (RMs) • NIST RM 8398 is available! – tinyurl.com/giabpilot – DNA isolated from large growth cell cultures – Stable, homogeneous – Best for regulated uses – DNA from same cell line at Coriell (NA12878) • New AJ and Asian Samples – Available from Coriell now – NIST RM available in 2016
  • 12.
    genomeinabottle.org Jan 2016 Workshop Thursday •Update and Roadmap • Breakouts – Analyses for PGP GIAB Trios – Reference Material Selection and Development • Breakout reports • Roadmap discussion Friday • Using GIAB Products for technology development, optimization, and demonstration – Experiences from the consortium • Steering committee
  • 13.
    genomeinabottle.org Steering Committee Meeting Topics •Future workshops • Format • Program committee? • Crafting a mission statement • Defining scope • Liaison with other efforts Current members – Marc Salit – Justin Zook – David Mittelman – Andrew Grupe – Michael Eberle – Steve Sherry – Deanna Church – Francisco De La Vega – Christian Olsen – Monica Basehore – Lisa Kalman – Christopher Mason – Elizabeth Mansfield – Liz Kerrigan – Leming Shi – Melvin Limson – Alexander Wait Zaranek – Nils Homer – Fiona Hyland – Steve Lincoln – Don Baldwin – Robyn Temple-Smolkin – Chunlin Xiao – Kara Norman – Luke Hickey
  • 14.
    genomeinabottle.org Agenda Monday • Breakfast andregistration • Welcome and Context Setting • NIST RM Update and Status Report • Charge to Working Groups • Coffee Break • Working Group Breakout Discussions • Lunch (provided) • Informal Working Group Reports • Coffee Break • Breakout Topical Discussions – Topic #1: Moving beyond the 'easy' variants and regions of the genome – Topic #2: Selecting future genomes for Reference Materials Tuesday • Breakfast and registration • Use cases: Experiences using the pilot Reference Material • Discussion of plans to release pilot Reference Material • Coffee Break • Working Group Breakout discussions • Lunch (provided) • Working Group leaders present plans and discussion • Steering committee Overview • First meeting of the Steering Committee (others adjourn) Please Note Slides will be made available on SlideShare after the workshop (see genomeinabottle.org). Tweets are welcome unless the speaker requests otherwise. Please use #giab as the hashtag.
  • 15.
    We are liaisingwith… • Illumina Platinum Genomes • CDC GeT-RM • Korean Genome Project • Genome Reference Consortium • 1000 Genomes SV group • CAP/CLIA • ABRF • FDA • SEQC • Global metrology system • Global Alliance for Genomics and Health Benchmarking Team • NCBI/CDC GeT-RM Browser • GCAT website
  • 16.
    NGS RM Project Slidecourtesy of Lisa Kalman, CDC
  • 17.
    Association of BiomolecularResource Facilities (ABRF) www.abrf.org Next Generation Sequencing Study Phase 2: DNA sequencing platforms Study Design and Launch Plan Slides courtesy of Chris Mason January 24, 2016
  • 18.
    Aims Create reference datasets - Sequence data from reference samples will be generated with intra- and inter-lab replication to model the likely range of performance that should be expected under normal service laboratory conditions. Test and create reference samples - Designated reference samples will be easily accessible to the community for self-evaluation by comparison to the reference data. Samples should be standardized, able to be stably reproduced over time, and suitable for development of new laboratory and bioinformatics methods. Data release and Immediate utility - Performance metrics and data will be developed for instrument platforms and sample preparation protocols that are deployed now or will be in the near future in core sequencing facilities. After QC, data will be released to the entire Genome in a Bottle (GIAB) and ABRF Consortia for use and preparation for submission as publications. ABRF NGS Phase II Study
  • 19.
    Samples and Platforms– All tested in triplicate across three distinct sites Platform Human DNA Bacterial DNA Illumina HiSeq X Ten A, B, C, C2, C2f Illumina HiSeq 4000 A, B, C Illumina HiSeq 2500 v4 1T A, B, C Illumina HiSeq 2500 v3 Rapid Run C Ste, Eco, Mil, P Illumina NextSeq 500 High Output C Illumina MiSeq Ste, Eco, Mil, P Life Tech Proton A, B, C exomes Ste, Eco, Mil, P Life Tech S5 A, B, C exomes Ste, Eco, Mil, P Life Tech PGM Ste, Eco, Mil, P Pacific Biosciences Ste, Eco, Mil, P Oxford Nanopore Ste, Eco, Mil, P maternal paternal son son (Coriell) A B C C2 Ste Eco Mil pool Human Trio Bacterial Isolates and Mixture ABRF NGS Phase II Study
  • 20.
    Reference DNA, TruSeq PCR-free350 FFPE DNA, TruSeq Nano FFPE DNA, TruSeq PCR-free KAPA libraries from sites a-b-c Ste Eco Mil pool maternal paternal son son (Coriell) %GC: 28 50 72 A B C C2 Personal Genome Project NIST Reference Human Genomes C2f Reference bacterial genomes TruSeq PCR-free 550 Ca Illumina (ILMN) - Samples ABRF NGS Phase II Study
  • 21.
  • 22.
    Sequencing Quality Control PhaseII (SEQC2) – An Introduction Slides courtesy of Weida Tong, Ph.D. Division of Bioinformatics and Biostatistics, NCTR/FDA 22
  • 23.
    Short reads vs longreads Detection power for rare mutation Detection accuracy for difficult genes Application scope of MiSeq Variants call (e.g., SNV, CNV, Indels) Assess the WGS accuracy and reproducibility for variants call by investigating the join effect of reads alignment pipelines, variants call methods and coverage as well as comparing the results from personal genome versus reference genome. Assess detection power of ultra-deep sequencing (TGS) for subclonal mutation and its dependency on bioinformatics and coverage. Assess the utility of MiSeq for (1) detection of subclonal mutation, (2) the difficult genes (e.g., HLA), and (3) the difficult variations (e.g., Indels) Assess the accuracy for some difficult genes that varies significantly due to complexity in their genomic regions (e.g. GC region) with specifically focused on HLA genes. Datasets: • Approaches: WGS and TGS • Platforms: Hiseq, PacBio, MiSeq, etc • Samples: TRIOs, NB, cell lines, etc Parameters: • Personal vs reference genome • Bioinformatics • Coverage SEQC2 Overview Assess short reads alone, long reads alone and their combination for genome assembly and subsequent variant calling in WGS.
  • 24.
    Trio Study Coverage/platform Notes Short readsLong reads SEQC2: HapMap Trio (European) 80x TBD Planned for both WGS and TGS; genotyping data and information from HapMap are available GIAB: Trio (Ashkenazim) Illumina 300x 69x (son), 30x (parents) This dataset is generated by Genome In A Bottle (GIAB) consortium. We closely work with GIAB to obtain the update information of this Trio and the GIAB leaders also participate in SEQC2. Complete Genomics BioNano Ion Torrent Moleculo SOLiD (WGS) SEQC2: Chinese Trio and test of LCL- germline 100x 50x Panned; the datasets will be provided by Dr. Leming Shi who is a part of SEQC2 leadership team. Three Trio Datasets 24
  • 25.
    Candidate NIST ReferenceMaterials Genome PGP ID Coriell ID NIST ID NIST RM # CEPH Mother/Daugh ter N/A GM12878 HG001 RM8398 AJ Son huAA53E0 GM24385 HG002 RM8391 (son)/RM8392 (trio) AJ Father hu6E4515 GM24149 HG003 RM8392 (trio) AJ Mother hu8E87A9 GM24143 HG004 RM8392 (trio) Asian Son hu91BD69 GM24631 HG005 RM8393 Asian Father huCA017E GM24694 N/A N/A Asian Mother hu38168C GM24695 N/A N/A
  • 26.
    NIST Microbial GenomicDNA Reference Materials Credit: Nate Olson
  • 27.
    Analysis process forMicrobial RMs Credit: Nate Olson
  • 28.
  • 29.
    Dataset AJ SonAJ Parents Chinese son Chinese parents NA12878 Illumina Paired- end X X X X X Illumina Long Mate pair X X X X X Illumina “moleculo” X X X X X Complete Genomics X X X X X Complete Genomics LFR X X X Ion exome X X X X BioNano X X X X 10X X X X PacBio X X X SOLiD single end X X X Illumina exome X X X X Oxford Nanopore X
  • 30.
  • 31.
    GIAB Analysis Group– New Data Sets Leaders • Francisco de la Vega – Stanford, TOMA Biosciences • Chris Mason – Weil Cornell Medical Center • Tina Graves – Washington University • Valerie Schneider – NCBI •and Justin and Marc Strategic Documents • Analysis Group Responsibilities: – https://docs.google.com/document/d/10e A0DwB4iYTSFM_LPO9_2LyyN2xEqH49OXH htNH1uzw/edit?usp=sharing • Analysis Milestones: – https://docs.google.com/spreadsheets/d/1Pj4nSz H742g40wJz2fA6f8kFtZYAToZpSZYVPiC5st4/edit?u sp=sharing • Analysis Methods – https://docs.google.com/spreadsheet s/d/1Je2g85H7oK6kMXbBOoqQ1FM NrvGnFuUJTJn7deyYiS8/edit?usp=sha ring • Analysis Plan: – https://drive.google.com/file/d/0B7Ao1qq JJDHQdnVEaVdqbWdEdkE/view?usp=shari ng • Collecting Data and analyses on GIAB FTP Site • Recruiting people to help with the work. Goal: Establish and distribute a set of authoritative benchmark variant calls of all types and sizes, as well as homozygous reference regions, on GIAB PGP trios
  • 32.
    GIAB Analysis Group– New Data Sets Types of analyses • SNPs/indels – NIST working on integration – 10X/moleculo/PacBio for difficult-to-map regions • Assembly – 2 de novo assemblies – Being used for SV calling Status • Structural variants – Candidate calls being generated by 15+ groups with >20 different algorithms and 6 datasets – 3+ integration methods – ~monthly calls • Long-range Phasing – 2 phased calls so far (CG LFR and 10X) – Integration methods needed • Methylation analyses Goal: Establish and distribute a set of authoritative benchmark variant calls of all types and sizes, as well as homozygous reference regions, on GIAB PGP trios
  • 33.
    genomeinabottle.org SNP/Indel Integration MethodUpdate • Implementing refined integration methods – Developed so others can readily reproduce results – Consistent results for all GIAB genomes – Simpler process taking advantage of best practices for each technology • Validating with released NA12878 RM data – Preliminary comparisons show minor changes • Application to PGP trios – Plan to analyze AJ trio by Q2 2016 – Release of NIST RMs in Q2 2016 – Develop calls for GRCh38
  • 34.
    genomeinabottle.org Data Release: Real-time,Open, Public Release Individual Datasets • Uploaded to GIAB FTP site as it is collected • Includes raw reads, aligned reads, and variant/reference calls Integrated High-confidence Calls • First develop SNP, indel, and homozygous reference calls • Then develop SV and non- SV calls • Released calls are versioned • Preliminary callsets will be made available to be critiqued
  • 35.
    GIAB AJ TrioHybrid PacBio/BioNano Assembly Hybrid (PacBio with BioNano) Input Assembly Notes # of Scaffolds N50 Max Total HG002 Falcon 248 22.7Mb 92.8Mb 2.38Gb Trio Falcon 210 29.3Mb 87.6Mb 2.32Gb Two Step Trio celera (child) + falcon (trio) 187 34.3Mb 98.0Mb 2.6Gb Credits: Ali Bashir, Jason Chin, Alex Hastie Pendleton et al, Nature Methods, 2015
  • 36.
  • 37.
    Proposed approach toform high- confidence SV (and non-SV) calls Generate Candidate Calls Compare/evaluate calls using Parliament/MetaSV/svclassify/others?; manual inspection Integrate new and revised calls; manual inspection Combine integrated calls; manual inspection; targeted experimental validation? August 30, 2015 January 2016 Plan in January 2016 Feb 2016 and beyond
  • 38.
    Deletion overlap summaryfor son By # of callsets # of callsets # of calls 1 3780 2 1391 3 859 4 574 5+ 344 By Technology Technology # of calls Illumina 3277 PacBio 5177 BioNano 812 CG 1758 Illumina/CG+PacBio 2318 Illumina/CG+BioNano 518 PacBio+BioNano 467 2+ technologies 2661 Converted all to bed; combined with bedtools multiinter; Calls within 50bps were merged
  • 39.
    Preliminary Confirmation ofSVs Integration results from AJ son Parliament: BMC Genomics, 2015, 16:286 (performed by Andrew Carroll, DNAnexus) MetaSV: Bioinformatics, 2015, 31:2741 (performed by Marghoob Mohiyuddin, Bina/Roche) • Parliament – Candidates from Illumina – Confirmed by PacBio and/or Illumina – ~50% in both technologies – ~4.5k deletions, 1k insertions – 85% of Genotypes consistent within Trio • MetaSV – Multiple types of evidence from Illumina MetaSV Total: 2809 Parliament Total: 5467 569 (20 %) 977 (18 %) MetaSV 2240 (80 %) Parliament 4490 (82 %) 50 % reciprocal overlap Some overlap within Parliament calls
  • 40.
    genomeinabottle.org GeT-RM Browser fromNCBI and CDC • http://www.ncbi.nlm.nih.gov/variation/tools/get-rm/ • Allows visualization of data underlying call each call
  • 41.
    genomeinabottle.org Uses of GIABNA12878 Oncology – Molecular and Cellular Tumor Markers “Next Generation” Sequencing (NGS) guidelines for somatic genetic variant detection www.bioplanet.com/gcat
  • 42.
    Global Alliance forGenomics and Health Benchmarking Task Team • Initial version of standardized definitions for performance metrics like TP, FP, and FN. • Continued development of sophisticated benchmarking tools • vcfeval – Len Trigg • hap.py – Peter Krusche • vgraph – Kevin Jacobs • Standardized intermediate and final file formats • Standardized bed files with difficult genome contexts for stratification • Simulating reads with non-SNP ClinVar variants to demonstrate importance of these tools • github.com/ga4gh/benchmarking -tools Next steps • Further analysis to demonstrate importance of sophisticated tools • Write manuscript about the team’s tools • Integrate vcfeval and hap.py to take advantage of strengths of each • Recommend “Best Practices” for benchmarking • Explore venues for making the team’s benchmarking process easier to use Progress
  • 43.
    Proposed Performance Metrics Definitions •Define TP/FP/FN/TN in 4 ways depending on required stringency of match: • Loose match: TP if within x-bp of a true variant • Allelle match: TP if ALT allele matches • Genotype match: TP if genotype and ALT allele match • Phasing match: TP if genotype, ALT allele, and phasing with nearby variants all match • True negatives are difficult to define because an infinite number of potential alleles exist
  • 44.
    genomeinabottle.org Global Alliance forGenomics and Health Benchmarking Task Team Credit: Rebecca Truty, Complete Genomics How should we interpret this complex variant on chr21?
  • 45.
    GA4GH Benchmarking ToolArchitecture Truth VCF Query VCF Comparison Engine vcfeval / vgraph / xcmp / bcftools / ... VCF-I Two-column VCF with TP/FP/FN annotations Quantification e.g. quantify / hap.py Stratification BED files Confident Call Regions VCF-R Two-column VCF with TP/FP/FN/UNK annotations Counts Credit: Peter Krusche https://github.com/ga4gh/benchmarking-tools
  • 46.
    Approaches to BenchmarkingVariant Calling • Well-characterized whole genome Reference Materials • Many samples characterized in clinically relevant regions • Synthetic DNA spike-ins • Cell lines with engineered mutations • Simulated reads • Modified real reads • Modified reference genomes • Confirming results found in real samples over time
  • 47.
    Challenges in BenchmarkingSmall Variant Calling • It is difficult to do robust benchmarking of tests designed to detect many analytes (e.g., many variants) • Easiest to benchmark only within high-confidence bed file, but… • Benchmark calls/regions tend to be biased towards easier variants and regions – Some clinical tests are enriched for difficult sites • Challenges with benchmarking complex variants near boundaries of high-confidence regions • Always manually inspect a subset of FPs/FNs • Stratification by variant type and region is important • Always calculate confidence intervals on performance metrics
  • 48.
    Particular Challenges inBenchmarking SV Calling • How to establish benchmark calls for difficult regions? • How to establish non-SV regions to assess FP rates? • Multiple dimensions of accuracy: – Predicted SV existence – Predicted SV type – Predicted size – Predicted breakpoints – Predicted exact sequence
  • 49.
    Acknowledgments • FDA –Elizabeth Mansfield • Many members of Genome in a Bottle – New members welcome! – Sign up on website for email newsletters GIAB Steering Committee – Marc Salit – Justin Zook – David Mittelman – Andrew Grupe – Michael Eberle – Steve Sherry – Deanna Church – Francisco De La Vega – Christian Olsen – Monica Basehore – Lisa Kalman – Christopher Mason – Elizabeth Mansfield – Liz Kerrigan – Leming Shi – Melvin Limson – Alexander Wait Zaranek – Nils Homer – Fiona Hyland – Steve Lincoln – Don Baldwin – Robyn Temple-Smolkin – Chunlin Xiao – Kara Norman – Luke Hickey
  • 50.
    For More Information www.genomeinabottle.org- sign up for general GIAB and Analysis Team google group emails github.com/genome-in-a-bottle – Guide to GIAB data & ftp www.slideshare.net/genomeinabottle www.ncbi.nlm.nih.gov/variation/tools/get-rm/ - Get-RM Browser Data: http://biorxiv.org/content/early/2015/09/15/026468 Global Alliance Benchmarking Team – https://github.com/ga4gh/benchmarking-tools Twice yearly public workshops – Winter at Stanford University, California, USA – Summer at NIST, Maryland, USA Justin Zook: jzook@nist.gov Marc Salit: salit@nist.gov
  • 51.
    GIAB Roadmap: Whereare we, Where are we going? • Reference Materials – Germline – Somatic • Informatics – Analysis of GIAB data – Benchmarking • Documentary Standards/Publications – Documentation of methods – Supporting Use