Komatsoulis internet2 global forum 2015
•Download as PPTX, PDF•
0 likes•219 views
NIH Data Initiatives: Harnessing Big (and small) Data to Improve Health Presentation at the internet2 Global Forum, April 28, 2015 Session NIH Perspectives
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (12)
Similar to Komatsoulis internet2 global forum 2015
Similar to Komatsoulis internet2 global forum 2015 (20)
Recently uploaded
Recently uploaded (20)
Komatsoulis internet2 global forum 2015
- 1. George A. Komatsoulis, Ph.D. National Center for Biotechnology Information (NCBI) National Library of Medicine National Institutes of Health U.S. Department of Health and Human Services
- 2. Mission: ”To seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability.” Composed of 27 Institutes and Centers Annual Budget = $30.3B 80% of NIH budget goes to about 50,000 grants
- 4. 1960 1970 1980 1990 2000 2010 2020
- 5. Sensor Stream = 500 EB/day Stores 69 TB/day Collection = 14 EB/day Store 1PB/day Total Data = 14 PB Store an average of 3.3TB/day for 10 years!
- 7. Launched to support biomedical data science research Support for multiple facets of data science: BD2K Centers Data and Software Discovery Standards and Interoperability Training and Workforce Development The Commons Led by Dr. Phil Bourne, NIH Associate Director for Data Science
- 8. Public Data Repositories Local Data U N I V E R S I T YU N I V E R S I T Y Locally Developed Software Publicly Available Software Local storage and compute resources
- 9. Is scalable and exploits new computing models Is more cost effective given digital growth Simplifies sharing digital research objects such as data, software, metadata and workflows Makes digital research objects more FAIR: Findable, Accessible, Interoperable and Reusable DOES NOT replace existing, well-curated databases Phil Bourne, 2014
- 10. TheCommons Digital Objects (with identifiers) Search (Indexed Metadata and API) Computing Platform OpenAPIs SoftwareEncapsulation
- 11. TheCommons Digital Objects (with identifiers) Search (Indexed Metadata and API) Computing Platform Commons Federation (Infrastructure) BD2K Centers DDICC (Search) Existing Resources Indexes Methods Content
- 13. Commons Federation (Infrastructure) Conformant Provider A Conformant Provider B Conformant Provider C
- 14. The Commons: Business Model Researcher Discovery Index The Commons Cloud Provider C Cloud Provider B Cloud Provider A NIH Provides Digital Objects Retrieves/Uses Digital Objects Option: Fund Providers to Support NIH Directed Resources Indexes Commons Provide Credits Uses Credits Finds Objects Commons Implemented as a federation of ‘conformant’ cloud providers and HPC environments Funded primarily by providing credits to investigators
- 15. Cost effective - Only pay for IT support used Drives competition – Better services at lower cost Supports Data sharing by driving science into the Commons Facilitates public-private partnership Scalable to most categories of data expected in the next 5 years.
- 16. Novelty: Never been tried, so we don’t have data about likelihood of success Cost Models: Predicated on stable or declining prices among providers True for the last several years, but we can’t guarantee that it will continue, particularly if there is significant consolidation in industry Service Providers: Predicated on service providers willing to make the investment to become conformant Market research suggests 3-5 providers within 2-3 months of program launch Persistence: The model is ‘Pay As You Go’ which means if you stop paying it stops going Giving investigators an unprecedented level of control over what lives (or dies) in the Commons
- 17. Investigator Reseller of Cloud Services The Commons Cloud Provider C Cloud Provider B Cloud Provider A Investigator Institution Directs reseller to distribute credits Instructs provider to put credits on investigator account 1 2 Review NIH 3 4 5 6 7 Approves Credit Request Requests Credits Uses credits Distributes Credits To Investigator
- 18. Minimum set of requirements for Business relationships (reseller, investigators) Interfaces (upload, download, manage, compute) Capacity (storage, compute) Networking and Connectivity Information Assurance Authentication and authorization Still need to work out details of how to manage approval of conformance A conformant cloud ≠ an IaaS provider Draft specification out for comment among vendors
- 19. Phase 0: Build the plumbing Phase 1: Pilot the model on a small number of investigators experienced with cloud computing, probably within the context of BD2K awards Phase 2: Open the Commons credit process to grantees from a subset of NIH Institutes and Centers Phase 3: Open the process to all NIH grantees
- 20. QA/QC Validation Aggregation Authoritative NCI Reference Data Set Data Coordinating Center NCI Genomic Data Commons (under development) NCI Clouds High Performance Computing Search/Retrieve Download Analysis
- 21. Secure Computational Capacity Pre-loaded Data Secure Computational Capacity Pre-loaded Data Secure Computational Capacity Pre-loaded Data NCI Genomics Consortium NCI Genomic Data Repositories
- 22. NIH Office of ADDS Vivien Bonazzi, Ph.D. Philip Bourne, Ph.D Michelle Dunn, Ph.D Mark Guyer, Ph.D. Jennie Larkin, Ph.D. Leigh Finnegan Beth Russell NCBI Dennis Benson, Ph.D. Alan Graeff David Lipman, MD Jim Ostell, Ph.D. Don Preuss Steve Sherry
Editor's Notes
- 1965 – Generation capacity < 100 aa’s/year/person => Dayhoff creates 1 base code to simplify computing in punch card era 1977 – Sanger and Maxam-Gilbert Sequencing invented. By mid 1980’s increase in production of 2 orders of magnitude (maybe 10-20K bases total 2-3K finished/year) 1986 – Development of dye based sequencing, ABI 370A 2000 bases/day/instrument by mid 1990’s 1996 – Development of DNA microarrays. 2 dye 100K chips => 200K/chip/day 2000’s- Next gen sequencing; 100M’s/day
- This has worked well for a long time, but: Every investigator has their own copy of the data! Every investigator needs the computational resources to do whatever calculation they want to do. Making locally developed software work outside of the local institution is often a challenge. Everyone likes the Broad Firehose, only Broad has made it work! Consider the TCGA Data Set (2.5 PB) Storage and Data Protection cost approximately $2,000,000 per year per copy Constant network updates at universities 2.5 PB = 20,000,000 Gb = 23 days at 10 Gb/sec Redundant computing environments Most HPC environments are either drastically over or under utilized This is an issue with more ‘normal’ sized data sets as well
- Mimimum Requirements: Business relationship is to allow distribution and billing of credits and to ensure that liability issues are resolved. Investigator that puts digital object in the commons is the one that retains the liability associated with its use. Interfaces – would need to be open, but not necessarily open-source. Requires support for basic operations. In addition, environment has to be open to all; so a private environment behind a university firewall won’t work. Identifiers and metadata: Tied together and together enable researchers to search for and find resources. Networking and Connectivity: Make sure that stuff is accessible, require connection to commodity internet and internet2, but key element from investigator point of view is a free egress tier for academics Environment is secure A&A: Must support inCommon because most NIH investigators have it. Minimizes hassle of granting access to collaborators across multiple platforms. Approval of clouds: Self certify vs. NIH certify vs. 3rd party certify. In early test cases, may simply say ‘FedRamped’ Cloud vs IaaS: Some IaaS (AWS comes to mind) may be uninterested in providing the ‘conformant’ layer but support other companies that provide these services using AWS backend. Already exemplars of this: Seven Bridges Genomics and the Cancer Genomics Cloud Pilots are all software layers over an IaaS provider.