The roles communities play in improving bioinformatics: better software, better algorithms, better data
•Download as ODP, PDF•
1 like•73 views
Discussing three community strategies to improve bioinformatics: collective open source development (biopython), critical assessment challenge (CAFA), and crowdsourcing (maize phenomics).
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to The roles communities play in improving bioinformatics: better software, better algorithms, better data
Similar to The roles communities play in improving bioinformatics: better software, better algorithms, better data (20)
More from Iddo
More from Iddo (20)
Recently uploaded
Recently uploaded (20)
The roles communities play in improving bioinformatics: better software, better algorithms, better data
- 1. The roles communities play in improving bioinformatics: better software, better algorithms, better data Iddo Friedberg Iowa State University @iddux iddo-friedberg.net
- 2. What is Bioinformatics? Applying computational tools to biological problems
- 3. DNA is Information ● DNA is composed of ordered nucleotides ● An alphabet of 4 characters ● Genome: the complete set of genetic material in an organism attttaaccgcaagcta...“The cellular machinery”
- 4. DNA is Information ● DNA is composed of ordered nucleotides ● An alphabet of 4 characters ● Genome: the complete set of genetic material in an organism attttaaccgcaagcta...“The cellular machinery” https://www.youtube.com/watch?v=xlPDEpimzB8
- 5. Genomes have many features positive regulation site negative regulatorsstop transcription genes REPs
- 6. Genome Sizes Vary Genome size in bases~bytes
- 7. Describing Genomes Genbank format FASTA format FASTQ format
- 8. Biopython: mostly parsers and data structures...
- 9. ... but many other functions Alignment, translation, transcription Visualization Interfaces with common bioinformatics programs Phylogenetics
- 10. Over 200 Contributors! https://github.com/biopython/biopython/blob/master/CONTRIB.rst 1999 2018 1350 citations (9/25/2018)
- 11. Managing Biopython ● A “coding culture” and standards are adhered to – Python PEP-8 style – Inline documentation – Unit testing ● Dual Licensing: Python license and BSD-3 ● Use version control and “social coding”: Github ● Distributions: Conda, pip, ...
- 12. Why Better Software? â—Ź Many contributors and users â—Ź Most contributors are users: eat your own dogfood â—Ź Low barriers to contribution and use â—Ź Quality breeds more quality: biopython incorporated in other software
- 14. How do we know what genes do? positive regulation site negative regulatorsstop transcription genes REPs
- 15. Experiments and Observations to Determine Gene Function
- 16. Most protein function are determined computationally, by homology inference â—Ź Homology: originating from a common ancestor â—Ź Often same function, but sometimes not â—Ź Sequence similarity is a proxy for homology
- 17. Homologs, same & different functions Locomotion
- 18. Homologs, same & different functions Locomotion Ground locomotion
- 19. Homologs, same & different functions Locomotion Ground locomotion Water locomotion
- 20. Inferring gene or protein homology by pairwise sequence alignment
- 21. Different methods to computationally predict function Lobb & Doxey, Current Opinion in Structural Biology 38 (2016)
- 22. There are Many Methods: Which is Best?
- 24. The Critical Assessment of Function Annotation “On the one hand, we have enormous “protein” databases that are replete with errors, wishful thinking, phantoms, and uncertainties. On the other, we have a tiny fraction of real proteins that have been studied in any depth.” –- Dan Graur
- 25. Standardizing “Function” with Gene Ontology Locomotion Ground locomotion Water locomotion
- 26. Standardizing “Function” with Gene Ontology
- 28. CAFA: The Critical Assessment of Functional Annotation CAFA is an ongoing community challenge for assessing protein function prediction methods.
- 31. Experimental annotations during this period are used as ground truth in evaluation 9/2016 2/2017 7/2017 CAFA3 announced with > 100,000 from 9 species Prediction phase ends 147 submission 62 groups Prediction phase ends 148 submissions 69 methods 1,755 Sequences collected as benchmarks for the assessment CAFA Timeline
- 32. CAFA is Growing
- 33. Benchmark Counts
- 35. Metrics DNA binding True function Predicted function
- 36. Nucleic Acid binding DNA binding Binding Metrics True function Predicted function
- 37. Nucleic Acid binding DNA binding Binding rRNA binding RNA binding Metrics True Positives : 2 False Positives: 2 False Negatives: 1 True function Predicted function
- 39. AUTHOR ZZZ MODEL 1 KEYWORDS sequence alignment. T96060020120 GO:0008270 0.80 T96060020120 GO:0003700 0.80 T96060020120 GO:0006351 0.80 T96060020119 GO:0005730 0.01 T96060020119 GO:0003676 0.07 T96060020119 GO:0005622 0.07 T96060020119 GO:0046872 0.07 T96060020118 GO:0008270 0.75 T96060020118 GO:0006351 0.68 T96060020118 GO:0003677 0.67 T96060020118 GO:0005634 0.67 T96060020118 GO:0006355 0.55 T96060020118 GO:0003700 0.34 Protein ID GO term Confidence
- 40. Precision Recall Curve 00 1 1 Recall Precision 2/4 2/3 â—Ź Results are calculated for each threshold of prediction confidence
- 41. Information content of a graph Clark & Radivojac. Bioinformatics. 29: i53 (2013).
- 42. Predicted: True: Information-theoretic similarity between graphs Missing Information: Remaining uncertainty: Clark & Radivojac. Bioinformatics. 29: i53 (2013).
- 43. Topological score vs. information theoretic score Precision Recall Remaining UncertaintyMissingInformation
- 45. Molecular Function Benchmark: all species Type: no knowledge Mode: full Metric: F-max
- 46. Molecular Function Benchmark: all species Type: no knowledge Mode: full Metric: S-min
- 47. Molecular Function Benchmark: all species Type: no knowledge Mode: full
- 48. Biological Process Benchmark: all species Type: no knowledge Mode: full Metric: F-max
- 49. Biological Process Benchmark: all species Type: no knowledge Mode: full Metric: S-min
- 50. Biological Process Benchmark: all species Type: no knowledge Mode: full
- 51. • Our ability to predict in MFO has improved • CAFA1: 0.60 • CAFA2: 0.60 • CAFA3: 0.70 Things We’ve Learned (top method) • Our ability to predict in BPO has improved, kinda... • CAFA1: 0.40 • CAFA2: 0.37 • CAFA3: 0.39
- 52. Are we doing better? Biological Process
- 53. Better Data
- 54. Phenomics A set of physical traits belonging ot a given organism Used in agriculture to improve crops Problem: obtain good gold standard data
- 58. Train a classifier using images labeled by different methods
- 59. How images are labeled
- 60. Fatigue: performance over time
- 61. Training data from the three crowds gives same classifier performance
- 62. Conclusions ● Communities of different types enhance and advance computational biology – Community software development (biopython) – Community challenges (CAFA) – Crowdsourcing ● Join or form a community!
- 63. Acknowledgements (CAFA) Iowa State University Huy Nguyen Md Nafiz Hamid Iddo Friedberg Indiana University Bloomington Yuxiang Jiang Predrag Radivojac University of Washington Timothy Bergquist Sean Mooney University of Pennsylvania Casey Greene European Bioinformatics Institute Maria Martin Claire O'Donovan George Georghiou Dartmouth College Deborah A. Hogan (Microbe Screen) Kimberley A. Lewis (Microbe Screen) Alex W. Crocker (Microbe Screen) Giovanni Bosco (Drosophila Screen) University of Illinois, Chicago Constance Jeffery
- 64. Corn Tassel Crowdsourcing Naihui Zhou, Zachary D. Siegel, Scott Zarecor Nigel Lee Darwin A. Campbell Carson M. Andorf Dan Nettleton Carolyn J. Lawrence-Dill Baskar Ganapathysubramanian Jonathan W. Kelly Iowa State University Presidential Interdisciplinary Research Initiative