MongoDB and research
•Download as PPT, PDF•
2 likes•1,081 views
The document discusses different data formats used for storing genomic and biological data, including flat text files, variant call format (VCF) files, and JSON documents. It provides examples of data stored in each format, such as genomic sequences, variant calls, and metadata. The formats vary in their structure, from unstructured text to structured fields. Properly formatting and organizing biological data allows for efficient analysis and sharing of information between researchers.
![MongoDB and research Jan Aerts, PhD Wellcome Trust Sanger Institute Hinxton, UK [email_address] @jandot](https://image.slidesharecdn.com/mongodbandresearch-100621094753-phpapp02/85/MongoDB-and-research-1-320.jpg)
Recommended
Recommended
More Related Content
Similar to MongoDB and research
Similar to MongoDB and research (20)
More from Jan Aerts
More from Jan Aerts (20)
Recently uploaded
Recently uploaded (20)
MongoDB and research
- 1. MongoDB and research Jan Aerts, PhD Wellcome Trust Sanger Institute Hinxton, UK [email_address] @jandot
- 2. Disclaimer 1
- 3. Disclaimer 2
- 4. Acknowledgments MongoDB community Karen Ambrose 10gen
- 6. transcriptomics genomics proteomics *omics
- 7. transcriptomics genomics proteomics *omics instantiationomics metabolomics spliceomics interactomics metallomics lipidomics orfeomics phenomics histomics
- 10. transitory
- 12. slow adoption of new technology (don't break anything that works)
- 13. data management = afterthought money
- 14. Who are the players?
- 27. 4. AceDB - A C aenorhabditis e legans d ata b ase object-oriented Author "Patel B" Full_name "Bala Patel" Laboratory CB Paper [cgc1011] Paper [cgc533] Mail "Laboratory of Molecular Biology" Mail "Hills Road, Cambridge" Fax "050 3456789" Paper [cgc533] Title "Yet more of those Genes" Journal "Cell Reports" Volume 3 Year 1993
- 29. Challenges in *omics - Where can MongoDB play a role?
- 30. explosion of data every researcher must be able to handle data
- 31. low stepping stone for bench-based scientists big data
- 34. Thank you! Questions? [email_address] @jandot http://saaientist.blogspot.com
Editor's Notes
- Not an expert. Reason will be explained further in the presentation.
- * personal ideas/opinions; not necessarily Sanger’s
- My background
- there are hurdles for adoption in academia
- Many people in institute; many ways of doing things + many tools
- Data is often transitory. Apart from the raw sequencing data (served by e.g. EBI): data can often be archived once paper is written.
- Because transitory. A one-off script that takes 5 minutes to write and a day to run is often preferable to one that takes a day to write and 5 minutes to run.
- Because we want to focus on the research, not the tools. If the available tools get the work done they will suffice.
- In many smaller labs: data management is not part of the initial grants. Is starting to change with the next-generation sequencing data.
- “ Genome hacker”: very broad. From guy-who-knows-how-to-record-macros-in-Word to hardcore mathematicians.
- "need IT support for heavier work": set up MongoDB server => what if need sharded cluster? => investment from IT "creating legacy": if it's something that will be used after you're gone (typical contract: postdoc = 3-5 yrs), you don't want to use a technology that is not supported or actively used within the organization “ often self-taught”!!!
- “ normalization?”: Overkill to try and persuade them to use databases if you have to teach them normal forms.
- What does the data look like?
- Very difficult to parse without custom libraries (bio*)
- “ //” => start of new record
- State of the art. Is tab-delimited, but not really.
- “ ##”: header “ #”: column headers INFO field: ‘;’-separated tag-value pairs (themselves separated with a ‘=‘) FORMAT field: necessary to know what is in the NA00001 column; colon-separated
- Not really tab-delimited anymore because too structured Self-taught => simple scripting languages!
- New technologies + existing technologies improved + decreasing cost of data generation
- Would benefit most. "bench-based scientists": - are more and more learning perl and working with tab-delimited files - to go from Exel to database: json looks more like how they think than having to cope with normalization steps in a relational database “ big data”: auto-sharding, mapreduce, …
- In-road into research: via department bioinformatician: constantly looking for new things Least effort of implementing and least costly if failure
- Focus is often on data-exchange => a lot of effort on exchange file formats