This document discusses science as a service (SaaS) and next-generation sequencing (NGS) data analysis. It summarizes challenges with exponential growth of NGS data, including data management, storage, analysis and sharing. It introduces Edge Bio's approach of distributing computational problems across cloud and HPC resources to avoid bottlenecks. Edge Bio provides full-service NGS analysis pipelines leveraging both commercial and open-source tools.

1. Closing the Gap in Time: From Raw Data to Real ScienceScience as a Service (ScaaS)Justin H. JohnsonDirector of BioinformaticsEdgeBioGaithersburg, MD

2. Edge Bio & MeSequencing and Bioinformatics ShopI am aBioinformaticianIT ProfessionalScientistThe lines are blurred…

3. NGS Exponential Growth Nature Biotechnology Volume 26 Number10 October2008

4. Sequencing is Free?Machines and VendorsLab Staffing, Integrations and LIMSProject DesignData Management and QCBioinformatics and Data AnalysisData Computes and StorageData Sharing

5. Machines and VendorsGnuBio

6. Ultra High Throughput + Lower Cost = Broader Applications

7. Bioinformatics Tools * BFAST - Blat-like Fast Accurate Search Tool. Written by Nils Homer, Stanley F. Nelson and Barry Merriman at UCLA.* Bowtie - Ultrafast, memory-efficient short read aligner. It aligns short DNA sequences (reads) to the human genome at a rate of 25 million reads per hour on a typical workstation with 2 gigabytes of memory. Uses a Burrows-Wheeler-Transformed (BWT) index. Link to discussion thread here. Written by Ben Langmead and Cole Trapnell. Linux, Windows, and Mac OS X.* BWA - Heng Lee's BWT Alignment program - a progression from Maq. BWA is a fast light-weighted tool that aligns short sequences to a sequence database, such as the human reference genome. By default, BWA finds an alignment within edit distance 2 to the query sequence. C++ source.* ELAND - Efficient Large-Scale Alignment of Nucleotide Databases. Whole genome alignments to a reference genome. Written by Illumina author Anthony J. Cox for the Solexa 1G machine.* Exonerate - Various forms of pairwise alignment (including Smith-Waterman-Gotoh) of DNA/protein against a reference. Authors are Guy St C Slater and Ewan Birney from EMBL. C for POSIX.* GenomeMapper - GenomeMapper is a short read mapping tool designed for accurate read alignments. It quickly aligns millions of reads either with ungapped or gapped alignments. A tool created by the 1001 Genomes project. Source for POSIX.* GMAP - GMAP (Genomic Mapping and Alignment Program) for mRNA and EST Sequences. Developed by Thomas Wu and Colin Watanabe at Genentec. C/Perl for Unix.* gnumap - The Genomic Next-generation Universal MAPper (gnumap) is a program designed to accurately map sequence data obtained from next-generation sequencing machines (specifically that of Solexa/Illumina) back to a genome of any size. It seeks to align reads from nonunique repeats using statistics. From authors at Brigham Young University. C source/Unix.* MAQ - Mapping and Assembly with Qualities (renamed from MAPASS2). Particularly designed for Illumina with preliminary functions to handle ABI SOLiD data. Written by Heng Li from the Sanger Centre. Features extensive supporting tools for DIP/SNP detection, etc. C++ source* MOSAIK - MOSAIK produces gapped alignments using the Smith-Waterman algorithm. Features a number of support tools. Support for Roche FLX, Illumina, SOLiD, and Helicos. Written by Michael Strömberg at Boston College. Win/Linux/MacOSX* MrFAST and MrsFAST - mrFAST & mrsFAST are designed to map short reads generated with the Illumina platform to reference genome assemblies; in a fast and memory-efficient manner. Robust to INDELs and MrsFAST has a bisulphite mode. Authors are from the University of Washington. C as source.* MUMmer - MUMmer is a modular system for the rapid whole genome alignment of finished or draft sequence. Released as a package providing an efficient suffix tree library, seed-and-extend alignment, SNP detection, repeat detection, and visualization tools. Version 3.0 was developed by Stefan Kurtz, Adam Phillippy, Arthur L Delcher, Michael Smoot, Martin Shumway, CorinaAntonescu and Steven L Salzberg - most of whom are at The Institute for Genomic Research in Maryland, USA. POSIX OS required.* Novocraft - Tools for reference alignment of paired-end and single-end Illumina reads. Uses a Needleman-Wunsch algorithm. Can support Bis-Seq. Commercial. Available free for evaluation, educational use and for use on open not-for-profit projects. Requires Linux or Mac OS X.* PASS - It supports Illumina, SOLiD and Roche-FLX data formats and allows the user to modulate very finely the sensitivity of the alignments. Spaced seed intial filter, then NW dynamic algorithm to a SW(like) local alignment. Authors are from CRIBI in Italy. Win/Linux.* RMAP - Assembles 20 - 64 bpIllumina reads to a FASTA reference genome. By Andrew D. Smith and ZhenyuXuan at CSHL. (published in BMC Bioinformatics). POSIX OS required.* SeqMap - Supports up to 5 or more bp mismatches/INDELs. Highly tunable. Written by Hui Jiang from the Wong lab at Stanford. Builds available for most OS's.* SHRiMP - Assembles to a reference sequence. Developed with Applied Biosystem'scolourspace genomic representation in mind. Authors are Michael Brudno and Stephen Rumble at the University of Toronto. POSIX.* Slider- An application for the Illumina Sequence Analyzer output that uses the probability files instead of the sequence files as an input for alignment to a reference sequence or a set of reference sequences. Authors are from BCGSC. Paper is here.* SOAP - SOAP (Short Oligonucleotide Alignment Program). A program for efficient gapped and ungapped alignment of short oligonucleotides onto reference sequences. The updated version uses a BWT. Can call SNPs and INDELs. Author is Ruiqiang Li at the Beijing Genomics Institute. C++, POSIX.* SSAHA - SSAHA (Sequence Search and Alignment by Hashing Algorithm) is a tool for rapidly finding near exact matches in DNA or protein databases using a hash table. Developed at the Sanger Centre by ZeminNing, Anthony Cox and James Mullikin. C++ for Linux/Alpha.* SOCS - Aligns SOLiD data. SOCS is built on an iterative variation of the Rabin-Karp string search algorithm, which uses hashing to reduce the set of possible matches, drastically increasing search speed. Authors are Ondov B, Varadarajan A, Passalacqua KD and Bergman NH.* SWIFT - The SWIFT suit is a software collection for fast index-based sequence comparison. It contains: SWIFT — fast local alignment search, guaranteeing to find epsilon-matches between two sequences. SWIFT BALSAM — a very fast program to find semiglobal non-gapped alignments based on k-mer seeds. Authors are Kim Rasmussen (SWIFT) and Wolfgang Gerlach (SWIFT BALSAM)* SXOligoSearch - SXOligoSearch is a commercial platform offered by the Malaysian based Synamatix. Will align Illumina reads against a range of Refseq RNA or NCBI genome builds for a number of organisms. Web Portal. OS independent.* Vmatch - A versatile software tool for efficiently solving large scale sequence matching tasks. Vmatch subsumes the software tool REPuter, but is much more general, with a very flexible user interface, and improved space and time requirements. Essentially a large string matching toolbox. POSIX.* Zoom - ZOOM (Zillions Of Oligos Mapped) is designed to map millions of short reads, emerged by next-generation sequencing technology, back to the reference genomes, and carry out post-analysis. ZOOM is developed to be highly accurate, flexible, and user-friendly with speed being a critical priority. Commercial. Supports Illumina and SOLiD data.Courtesy of SeqAnsers.com

8. Experimental Design ConsiderationsSequencing Platform in Use

9. Choice of Library Construction

10. Depth of coverage

11. Re$ources

12. Number of Replicates

13. Number of Samples and Control

14. Etc…The Sky Isn’t Falling

15. Its Building…11696384100,000350,000,000Tomorrow?

16. …and distributingDistributing the ProblemExchanging DataRefreshing Data

17. How do we avoid the Perfect Storm?http://www.flickr.com/photos/nationalmaritimemuseum/3115298137/

18. Life Vests?ScienceasaServiceSaasIaasPaasHaasDaas

19. Edge Bio

20. You can’t do this alone, neither can we.Elaine Mardis, Ph.D.Co-Director, Genome Sequencing CenterWashington University School of MedicineSam Levy, Ph.D.Director of Genomic SciencesProfessor of Translational Genomics & Human Genomic MedicineScripps Translational Science InstituteScripps HealthScripps Research InstituteMichael Zody, Ph.D.Chief TechnologistBroad Institute of MITKen Dewar, Ph.D.Assistant ProfessorMcGill University and Quebec GenomeSteven Salzberg, Ph.D.Director, Center for Bioinformatics and Computational BiologyUniversity of MarylandGabor Marth, Ph.D.Professor of BioinformaticsBoston CollegeElliott Margulies, Ph.D.InvestigatorGenome Informatics SectionNational Human Genome Research InstituteNational Institutes of HealthScientific Advisory Board

21. EdgeBio

22. Edge Bio

23. BioinformaticsCloud Computing (Iaas, PaaS)Amazon, Google, OthersNGS Software and AlgorithmsCommercial (CLC) and Open SourceFrameworks(cloud)Biolinux, Hadoop and ChefData Sharing and Standards (GSC/M5)

24. Cloud

25. CloudNot a talk on cloud computing…Either you already know what it is – OR – Other can do it more justice with more time.The 4 dollar genome.

26. An Academic ProblemHave 100K+ free computes avail (Teragrid)Can’t get to them easilyHave unlimited pool of computes (Amazon)Can’t figure out how to pay for themIf you have figured those out…Traditional IssuesPipes too smallSecurity

27. Edge Science Gateway (ESG)Easily leverage any source at the back endCloudTeraGridInternal HPC ClusterInternal cloud (Eucalyptus)Commercial partners provide thin clients layerCLC Bio

28. Edge Science Gateway (ESG)Plug and Play ToolsChef and (cloud)Bio-LinuxReal people, not services underneath it all.IFX and IT consulting

29. Edge Exome Analysis PipelineBase Call, Quality FilterMapGenomeExon junctionsVariant AnalysisSNPINDELVariant AnnotationRefseq (etc)dbSNP/100GenomesSIFT, PolyPhenOMIMRepeat DatabaseVisualizationFunctional and Structural Analysis

30. Sometimes money isn’t enoughTraditional Infrastructure or New Era CloudsStill costs, whether up front or on back endAlways want to minimize costFrameworks and Algorithms R&DNGS Software and Algorithmscloud(Bio-Linux)HadoopChef

31. NGS Software and AlgorithmsBest of Breed though pluggable architectureMake them faster…Map Reduce Blast & GPU Enable BlastSIMD Accelerated HMMR, ClustalW, SW (CLC Bio)…or run them less.Clustering algorithmsEfficient data refreshesSharing of results

32. FrameworksCloud(BioLinux)Easily deploy others software though virtualizationChef (an Intro)Open Source configuration management for your entire infrastructureMPI and HadoopSpeed up and I/O issue resolution with MPICloudBurst – mapReduce based NGS mapping

33. Edge BioServ Services Project goals and timelinesNumber of samplesNumber of reads/tags per sampleExperiment and Project DesignSample Preparation Library PreparationSample Capture Library ConstructionSample QC and QuantificationAdaptor ligation ProjectWorkflowAmplification & SequenceFragment amplificationNext Generation Sequencing run Align sequence to reference genomeor RNA databaseSecondary and Tertiary AnalysisData Analysis

34. Real World Examples1500+ Sample Epigenetic StudyChallengesTracking

35. QC

36. DeliverySolutionMix of Commercial and Open Tools

37. Geospiza

38. CLC Bio

39. Customs Algorithms

40. HPC and Storage

41. Onsite 200 TB NAS

42. S3 for Backup (Soon for Delivery)Real World ExamplesHundreds of exomes over the next 6 months.ChallengesPipeline for mapping and variant analysisSolutionMix of commercial, in house, and open source tools.

43. Edge Exome Analysis PipelineBase Call, Quality FilterMapGenomeExon junctionsVariant AnalysisSNPINDELVariant AnnotationRefseq (etc)dbSNP/100GenomesSIFT, PolyPhenOMIMRepeat DatabaseVisualizationFunctional and Structural Analysis

44. EdgeBioExon Coverage StatisticsHow well is the exome covered?~3 Quads on SOLiD 3 Plus <1% of exons have 0 coverage

45. 3% of exons have 0-5x coverage

46. 5% of exons have 0-10x coverage

47. 95% of exons have >10x coverage~1 Quad on SOLiD 3 Plus <2% of exons have 0 coverage

48. 10% of exons have 0-5x coverage

49. 17% of exons have 0-10x coverage

50. 83% of exons have >10x coverageReal World ExamplesTranscriptomes from microbes to mice.ChallengesRapidly evolving fieldVisualizationSolutionsAgain leveraging tools across the board via ESGPlug and Play Best of Breed.

51. Visualization

52. Re$ourcesQualityCostExpediency

53. Re$ourcesQualityCostExpediency

54. Re$ourcesQualityCostExpediency

55. Re$ourcesQualityCostExpediency

56. Re$ourcesQualityCostExpediency

57. Questions?Thank YouTwitter: @BioInfo & @EdgeBioEmail: jjohnson@edgebio.com

59. …Yet1000 Genomes+ 1500