The document describes a study that aimed to validate insertions and deletions (INDELs) identified in the Genome in a Bottle (GIAB) reference standard by comparing calls made by FreeBayes and GATK variant callers. Researchers analyzed sequencing data from the NA12878 genome using the two callers and GIAB, generating 7 variant lists. They selected 150 random INDELs from each list and designed primers to amplify the regions, then validated the variants using MiSeq sequencing. The results helped improve understanding of the variant callers and the GIAB standard for INDEL calls.

1. Validating INDELs in the Genome In A Bottle
Reference Genome Standard
Cameron Locker, Sarah Imawalle, Natalie Bir, Vijay Nadella, Ben Kelly, Harkness Kuck, James Fitch, Peter White
The Biomedical Genomics Core & The Center for Microbial Pathogenesis, The Research Institute at Nationwide Children's
Hospital, Columbus, OH
Methods
The National Institute of Science and Technology (NIST) produced the GIAB with their
own analysis of the NA12878 genome. The variant caller programs FREEBAYES and
GATK were used on a 30X coverage set of the NA12878 genome sequence data using our
in house data analysis pipeline “Churchill". The three variant lists were compared to
identify the variants in common, against those that were unique to each data set (Figure 6)
ResultsIntroduction
Goal: Validate the NIST Genome In A Bottle GIAB standard against two different
variant callers
• FREEBAYES (FB)
• Genome Analysis Tool Kit (GATK)
Approach:
• Identify the variants unique to each
• Develop a high throughput sequencing approach using the MiSeq
• Validate sequencing results to determine if the variant exists.
Figure 2: Sample variant in which all groups
correctly identified a change, but reported it
in different ways. Freebayes describes the
change as a block substitution, GATK as an SNP,
insertion and deletion, and GIAB as four single
nucleotide polymorphisms (SNP).
Figure 1: Output of the program IGVtools from the Broad Institute. Allows viewing of multiple variant
calls at the same location. All three of the groups are shown, differences described in Figure 2 and Table 1.
Table 1: Further explanation of the differences in
reporting among the groups. Individual base
changes described. All groups correctly identified
the change and the location, but differed when
describing how the change occurred.
Figure 3: Work flow for the identification, isolation and laboratory preparation of variants. The program RTGtools
was used to parse out the differences between the variant lists. By using a range for location it can filter out variants like the
one described in Figure 1 where despite describing the same variant, the method to do so was different. The program
Primer3 was used to design primers to be ordered for MiSeq analysis. PCR was used to amplify the desired sequences. They
were loaded into pools for analysis by the MiSeq data.
Figure 5: Fastq analysis of MiSeq data. The program
FLASh was used to merge the paired end reads. Another
program, cutAdapt, was used to remove the primer ends of the
sequence. Finally the sequence was compared to the expected
variant it was designed for to determine if the variant was
indeed valid.
Using RTGtools to properly parse out the difference between the variant callers, seven
variant lists were generated . These variants include both single nucleotide polymorphisms
(SNPs) as well as insertions and deletions (INDELs) (Table 2).
Figure 6: Distribution of variants among the cross-section of GIAB database and variant callers FB and GATK.
Table 2: INDEL counts for each of the cross-sectional lists. 150 variants (if the files were large enough) were
randomly selected from each file. The breakdown of the 150 is 70 random insertions, 70 random deletions, and 10
random multi allelic variants.
Figure 4: Gel process during the ‘pool’ phase. Each
band represents the DNA of the variant loaded into that
well. The further the band goes, the lighter the sequence
is. Multiple bands can indicate the variant is
heterozygous such that there are unique sequences from
both chromosomes.
Validation of chr2: 79133830 CACACACACCCTAT>C GIAB variant
GGTGGCACAGATAAGGACACAGTAGTCATGAGCTTTTGCCCACAGTAAACTGGATGATTACTGAAAGAAAGGGAGGCTGACAAGGAGAG
CCTGTGATTAAGGTAGAAAAGGTTTTCAACCAGGGCCCTTTCAAGCAGCACTGAGAACATTTCAGCTTCTTCCTTCCAGCCTTGGAGAG
GAAAGTACACACACACACACACACACACACACACACACACACACACACACACACACCCTATCTTTTTTTTTCTTTTGACTAAAGACAGA
TGATGACATGGTTGACCAGTATTCACACACACTCAAAGAAGTTAAATGCTTTTTAGCTGACAGTCATCTCAAATCCTTCTAGAAAACAA
CACAAAATACTTTATGTGATTTGCTGGTCACTTCACTGTTTAGCCC
Figure 7: Validating MiSeq results for a deletion on chromosome 2. In yellow, the forward read, in blue, the reverse
read. The overlap between the two is colored in green. The deletions from this region of chromosome 2 are found in
purple.
GIAB
Freebayes
GATK
Block Substitution
Insertion and Deletion
SNPs
Variant List Ref > Alt
GTACCA>GCGGCG
Type of
Variant
Freebayes TACCA > CGGCG Block Substitution
GATK G> GCGG
GTAC > G
A > G
Insertion and
Deletion
GIAB T > C
A > G
C > G
A > G
SNP
All correctly report the same result
Acknowledgements
This work was supported by Nationwide Children’s Hospital Biomedical Genomics core.
PCR, Gel data collection, and MiSeq machine operation done by Sarah Imawalle.
Discussion
Validating the existence of these variants:
• Achieve a better understanding of the variant callers used
• Help improve the GIAB standard
Moving Forward:
• Continue to validate INDELs among the seven variant lists
• Test new variant callers
• Improve the MiSeq INDEL validation pipeline