SlideShare a Scribd company logo
1 of 11
Judicious use of custom
development in an open source
component architecture
Sky Bristol1
Natalie Latysh1
Dell Long2
Steve Tekell2
Jeff Allen2
U.S. Department of the Interior
U.S. Geological Survey
1. USGS Core Science Analytics,
Synthesis and Libraries
2. USGS Fort Collins Science
Center
AGU Fall Meeting 2014
Talking Points
• ScienceBase – inspiration and a brief history
• Development methodology
• Information architecture
• Cyberinfrastructure architecture
• API first
• Lessons learned and motivators for future
• Open invitation (and invite us to your party
too)
AGU Fall Meeting 2014
myUSGS Data
Explorer/Data
Uploader
Scientific Data
Catalog/Compr
ehensive
Science Catalog
ScienceBase 1.0
and then 2.0
ScienceBase-
Constellation
Collaborative
tools and simple
file upload
Metadata
Cataloging &
Research Item
Concept
Digital Repository
& Research Item
Faceting
Federation of
Disparate Data
Schemas
“In the research process, we need more than just a big catalog of data. We need all
of the other important information connected to our work – published papers,
manuscripts, software, and information about people, labs, projects, and others in
our field.”
Inspiration and History
AGU Fall Meeting 2014
Agile Development
AGU Fall Meeting 2014
Flexible Information Architecture
AGU Fall Meeting 2014
Cyberinfrastructure Overview
AGU Fall Meeting 2014
Manage and Contribute
Register
Find it at the USGS
Find it at the DOI
Find it at Data.gov
Find it in a Data.gov community
(climate.data.gov)
Find it in other communities
(GeothermalData.org)
Open Data and USGIN Methods (“abridged”)
AGU Fall Meeting 2014
API First
AGU Fall Meeting 2014
Lessons Learned
• We encounter some new type of data just about every
week and need to account for a lot more variability.
• It’s pretty expensive to run an institutional repository,
but we have it to do.
• Baking security in from the ground up adds quite a few
constraints to the ideal architecture.
• We can’t please everyone all the time, but if we keep
building great platforms then people come back.
• APIs (and everything else, for that matter) need to be
driven by real, specific use cases to be usable.
• R&D, engineering, and operations are all different and
need different management methods.
AGU Fall Meeting 2014
Community Framework and
Platform vs. Agency System
AGU Fall Meeting 2014
Questions
www.sciencebase.gov
sciencebase@usgs.gov
Natalie Latysh (nlatysh@usgs.gov)
Sky Bristol (sbristol@usgs.gov)

More Related Content

What's hot

20130525 toledo-odin-sergio ruiz
20130525 toledo-odin-sergio ruiz20130525 toledo-odin-sergio ruiz
20130525 toledo-odin-sergio ruizFESABID
 
Science base usage analysis - AGU2016 - in21d08
Science base usage analysis - AGU2016 - in21d08Science base usage analysis - AGU2016 - in21d08
Science base usage analysis - AGU2016 - in21d08Sky Bristol
 
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...MongoDB
 
Hack reduce introduction
Hack reduce introductionHack reduce introduction
Hack reduce introductionmontrealouvert
 
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speech
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speechBig Data Applied, Data Warehouse Institute St. Louis December 2013 speech
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speechDavid Strom
 
Top 5 AI and Deep Learning Stories - November 9, 2018
Top 5 AI and Deep Learning Stories - November 9, 2018Top 5 AI and Deep Learning Stories - November 9, 2018
Top 5 AI and Deep Learning Stories - November 9, 2018NVIDIA
 
Guest Lecture on Big Data in Business,
Guest Lecture on Big Data in Business, Guest Lecture on Big Data in Business,
Guest Lecture on Big Data in Business, saravana krishnamurthy
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Redis Labs
 
Big data – An Introduction, July 2013
Big data – An Introduction, July 2013Big data – An Introduction, July 2013
Big data – An Introduction, July 2013Peter Morgan
 
Infrastructure - a journey from datacentres to cloud
Infrastructure - a journey from datacentres to cloudInfrastructure - a journey from datacentres to cloud
Infrastructure - a journey from datacentres to cloudEqual Experts
 
Business Insight
Business InsightBusiness Insight
Business InsightMicrosoft
 
#19sverificado : How Data Analytics helped put order in chaos in Mexico's ...
#19sverificado :  How Data Analytics helped  put order in chaos in  Mexico's ...#19sverificado :  How Data Analytics helped  put order in chaos in  Mexico's ...
#19sverificado : How Data Analytics helped put order in chaos in Mexico's ...Gustavo Rene Antunez
 
Reproducibility and Dataverse
Reproducibility and DataverseReproducibility and Dataverse
Reproducibility and Dataversephilipdurbin
 
Cassandra for bay area big data developer
Cassandra for bay area big data developerCassandra for bay area big data developer
Cassandra for bay area big data developer19scottmiller
 
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...exponential-inc
 
Big data perspective solution & technology
Big data perspective solution & technologyBig data perspective solution & technology
Big data perspective solution & technologyPankaj Khattar
 
#4 FAIR - Keith Russell
#4 FAIR - Keith Russell #4 FAIR - Keith Russell
#4 FAIR - Keith Russell ARDC
 

What's hot (20)

20130525 toledo-odin-sergio ruiz
20130525 toledo-odin-sergio ruiz20130525 toledo-odin-sergio ruiz
20130525 toledo-odin-sergio ruiz
 
Science base usage analysis - AGU2016 - in21d08
Science base usage analysis - AGU2016 - in21d08Science base usage analysis - AGU2016 - in21d08
Science base usage analysis - AGU2016 - in21d08
 
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
 
Hack reduce introduction
Hack reduce introductionHack reduce introduction
Hack reduce introduction
 
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speech
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speechBig Data Applied, Data Warehouse Institute St. Louis December 2013 speech
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speech
 
Top 5 AI and Deep Learning Stories - November 9, 2018
Top 5 AI and Deep Learning Stories - November 9, 2018Top 5 AI and Deep Learning Stories - November 9, 2018
Top 5 AI and Deep Learning Stories - November 9, 2018
 
Guest Lecture on Big Data in Business,
Guest Lecture on Big Data in Business, Guest Lecture on Big Data in Business,
Guest Lecture on Big Data in Business,
 
ieee cloud 2015 keynote talk
ieee cloud 2015 keynote talkieee cloud 2015 keynote talk
ieee cloud 2015 keynote talk
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
 
Big data – An Introduction, July 2013
Big data – An Introduction, July 2013Big data – An Introduction, July 2013
Big data – An Introduction, July 2013
 
Infrastructure - a journey from datacentres to cloud
Infrastructure - a journey from datacentres to cloudInfrastructure - a journey from datacentres to cloud
Infrastructure - a journey from datacentres to cloud
 
Business Insight
Business InsightBusiness Insight
Business Insight
 
#19sverificado : How Data Analytics helped put order in chaos in Mexico's ...
#19sverificado :  How Data Analytics helped  put order in chaos in  Mexico's ...#19sverificado :  How Data Analytics helped  put order in chaos in  Mexico's ...
#19sverificado : How Data Analytics helped put order in chaos in Mexico's ...
 
Reproducibility and Dataverse
Reproducibility and DataverseReproducibility and Dataverse
Reproducibility and Dataverse
 
Cassandra for bay area big data developer
Cassandra for bay area big data developerCassandra for bay area big data developer
Cassandra for bay area big data developer
 
Tracking data lineage at Stitch Fix
Tracking data lineage at Stitch FixTracking data lineage at Stitch Fix
Tracking data lineage at Stitch Fix
 
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
 
Big data perspective solution & technology
Big data perspective solution & technologyBig data perspective solution & technology
Big data perspective solution & technology
 
Books neended
Books neendedBooks neended
Books neended
 
#4 FAIR - Keith Russell
#4 FAIR - Keith Russell #4 FAIR - Keith Russell
#4 FAIR - Keith Russell
 

Viewers also liked

Self learning material
Self learning materialSelf learning material
Self learning materialSangam Singh
 
Pgt432 PBLproject
Pgt432 PBLprojectPgt432 PBLproject
Pgt432 PBLprojectourslides
 
2009年中国数学会学术年会与会者通讯录
2009年中国数学会学术年会与会者通讯录2009年中国数学会学术年会与会者通讯录
2009年中国数学会学术年会与会者通讯录Xu jiakon
 
Доски объявлений
Доски объявленийДоски объявлений
Доски объявленийAndreygold
 
Microsoft opensource
Microsoft opensourceMicrosoft opensource
Microsoft opensourceCDS
 
Microsoft opensource
Microsoft opensourceMicrosoft opensource
Microsoft opensourceCDS
 
Enfermedades respiratorias en niños de 0 a un
Enfermedades respiratorias en niños de 0 a unEnfermedades respiratorias en niños de 0 a un
Enfermedades respiratorias en niños de 0 a uneveryn
 
懷生國中與綠色教室
懷生國中與綠色教室懷生國中與綠色教室
懷生國中與綠色教室family
 
Airiti books user_guide
Airiti books user_guideAiriti books user_guide
Airiti books user_guideairitiBooks
 
ABC's of Talent Selection
ABC's of Talent Selection ABC's of Talent Selection
ABC's of Talent Selection Kyle Deweerdt
 

Viewers also liked (20)

Self learning material
Self learning materialSelf learning material
Self learning material
 
Pgt432 PBLproject
Pgt432 PBLprojectPgt432 PBLproject
Pgt432 PBLproject
 
2009年中国数学会学术年会与会者通讯录
2009年中国数学会学术年会与会者通讯录2009年中国数学会学术年会与会者通讯录
2009年中国数学会学术年会与会者通讯录
 
Monografia de cuapiaxtla
Monografia de cuapiaxtlaMonografia de cuapiaxtla
Monografia de cuapiaxtla
 
Communicating health and safety messages
Communicating health and safety messagesCommunicating health and safety messages
Communicating health and safety messages
 
Доски объявлений
Доски объявленийДоски объявлений
Доски объявлений
 
Microsoft opensource
Microsoft opensourceMicrosoft opensource
Microsoft opensource
 
Microsoft opensource
Microsoft opensourceMicrosoft opensource
Microsoft opensource
 
Tales2 go
Tales2 goTales2 go
Tales2 go
 
Enfermedades respiratorias en niños de 0 a un
Enfermedades respiratorias en niños de 0 a unEnfermedades respiratorias en niños de 0 a un
Enfermedades respiratorias en niños de 0 a un
 
懷生國中與綠色教室
懷生國中與綠色教室懷生國中與綠色教室
懷生國中與綠色教室
 
Airiti books user_guide
Airiti books user_guideAiriti books user_guide
Airiti books user_guide
 
Hartstichting 2009
Hartstichting 2009Hartstichting 2009
Hartstichting 2009
 
Agriprom Dairy
Agriprom DairyAgriprom Dairy
Agriprom Dairy
 
ABC's of Talent Selection
ABC's of Talent Selection ABC's of Talent Selection
ABC's of Talent Selection
 
Accessible Gardening ~ Surf Coast Shire Council
Accessible Gardening ~ Surf Coast Shire CouncilAccessible Gardening ~ Surf Coast Shire Council
Accessible Gardening ~ Surf Coast Shire Council
 
Alternativa #somalternativa
Alternativa #somalternativaAlternativa #somalternativa
Alternativa #somalternativa
 
Binder1
Binder1Binder1
Binder1
 
Pravdepodobnost
PravdepodobnostPravdepodobnost
Pravdepodobnost
 
Sekai design impact
Sekai design impactSekai design impact
Sekai design impact
 

Similar to Judicious use of custom development in an open source component architecture

Working towards Sustainable Software for Science (an NSF and community view)
Working towards Sustainable Software for Science (an NSF and community view)Working towards Sustainable Software for Science (an NSF and community view)
Working towards Sustainable Software for Science (an NSF and community view)Daniel S. Katz
 
Tools für das Management von Forschungsdaten
Tools für das Management von ForschungsdatenTools für das Management von Forschungsdaten
Tools für das Management von ForschungsdatenHeinz Pampel
 
Using splunk for_big_data
Using splunk for_big_dataUsing splunk for_big_data
Using splunk for_big_dataAccenture
 
Big Data Curricula at the UW eScience Institute, JSM 2013
Big Data Curricula at the UW eScience Institute, JSM 2013Big Data Curricula at the UW eScience Institute, JSM 2013
Big Data Curricula at the UW eScience Institute, JSM 2013University of Washington
 
Transforming Networking within ESIP using ResearchBit
Transforming Networking within ESIP using ResearchBitTransforming Networking within ESIP using ResearchBit
Transforming Networking within ESIP using ResearchBitErin Robinson
 
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...datacite
 
What is eScience, and where does it go from here?
What is eScience, and where does it go from here?What is eScience, and where does it go from here?
What is eScience, and where does it go from here?Daniel S. Katz
 
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...hsuleslie
 
Introduction to UC San Diego’s Integrated Digital Infrastructure
Introduction to UC San Diego’s Integrated Digital InfrastructureIntroduction to UC San Diego’s Integrated Digital Infrastructure
Introduction to UC San Diego’s Integrated Digital InfrastructureLarry Smarr
 
20160414 23 Research Data Things
20160414 23 Research Data Things20160414 23 Research Data Things
20160414 23 Research Data ThingsKatina Toufexis
 
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...Kerstin Lehnert
 
SGCI Science Gateways Landscape in North America
SGCI Science Gateways Landscape in North AmericaSGCI Science Gateways Landscape in North America
SGCI Science Gateways Landscape in North AmericaSandra Gesing
 
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...Repository Fringe
 
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...SURFevents
 
2016 Ocean Sciences Meeting tutorial
2016 Ocean Sciences Meeting tutorial2016 Ocean Sciences Meeting tutorial
2016 Ocean Sciences Meeting tutorialJosh Young
 
Global registries initiative frumkin omodei
Global registries initiative frumkin omodeiGlobal registries initiative frumkin omodei
Global registries initiative frumkin omodeiASIS&T
 
Guy avoiding-dat apocalypse
Guy avoiding-dat apocalypseGuy avoiding-dat apocalypse
Guy avoiding-dat apocalypseENUG
 
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science, a Digital Research...
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science,  a Digital Research...Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science,  a Digital Research...
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science, a Digital Research...Carole Goble
 

Similar to Judicious use of custom development in an open source component architecture (20)

Working towards Sustainable Software for Science (an NSF and community view)
Working towards Sustainable Software for Science (an NSF and community view)Working towards Sustainable Software for Science (an NSF and community view)
Working towards Sustainable Software for Science (an NSF and community view)
 
Tools für das Management von Forschungsdaten
Tools für das Management von ForschungsdatenTools für das Management von Forschungsdaten
Tools für das Management von Forschungsdaten
 
Using splunk for_big_data
Using splunk for_big_dataUsing splunk for_big_data
Using splunk for_big_data
 
Big Data Curricula at the UW eScience Institute, JSM 2013
Big Data Curricula at the UW eScience Institute, JSM 2013Big Data Curricula at the UW eScience Institute, JSM 2013
Big Data Curricula at the UW eScience Institute, JSM 2013
 
Transforming Networking within ESIP using ResearchBit
Transforming Networking within ESIP using ResearchBitTransforming Networking within ESIP using ResearchBit
Transforming Networking within ESIP using ResearchBit
 
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
 
What is eScience, and where does it go from here?
What is eScience, and where does it go from here?What is eScience, and where does it go from here?
What is eScience, and where does it go from here?
 
BLC & Digital Science: Mark Hahnel, Figshare
BLC & Digital Science: Mark Hahnel, FigshareBLC & Digital Science: Mark Hahnel, Figshare
BLC & Digital Science: Mark Hahnel, Figshare
 
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
 
Introduction to UC San Diego’s Integrated Digital Infrastructure
Introduction to UC San Diego’s Integrated Digital InfrastructureIntroduction to UC San Diego’s Integrated Digital Infrastructure
Introduction to UC San Diego’s Integrated Digital Infrastructure
 
20160414 23 Research Data Things
20160414 23 Research Data Things20160414 23 Research Data Things
20160414 23 Research Data Things
 
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
 
SGCI Science Gateways Landscape in North America
SGCI Science Gateways Landscape in North AmericaSGCI Science Gateways Landscape in North America
SGCI Science Gateways Landscape in North America
 
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
 
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
 
2016 Ocean Sciences Meeting tutorial
2016 Ocean Sciences Meeting tutorial2016 Ocean Sciences Meeting tutorial
2016 Ocean Sciences Meeting tutorial
 
Data Science and Urban Science @ UW
Data Science and Urban Science @ UWData Science and Urban Science @ UW
Data Science and Urban Science @ UW
 
Global registries initiative frumkin omodei
Global registries initiative frumkin omodeiGlobal registries initiative frumkin omodei
Global registries initiative frumkin omodei
 
Guy avoiding-dat apocalypse
Guy avoiding-dat apocalypseGuy avoiding-dat apocalypse
Guy avoiding-dat apocalypse
 
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science, a Digital Research...
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science,  a Digital Research...Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science,  a Digital Research...
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science, a Digital Research...
 

More from Sky Bristol

CanyonViews.pptx
CanyonViews.pptxCanyonViews.pptx
CanyonViews.pptxSky Bristol
 
USGS research infrastructure - AGU2016 - in13 e
USGS research infrastructure - AGU2016 - in13 eUSGS research infrastructure - AGU2016 - in13 e
USGS research infrastructure - AGU2016 - in13 eSky Bristol
 
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...Sky Bristol
 
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...Sky Bristol
 
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...Sky Bristol
 
ScienceBase and CINERGI - thoughts
ScienceBase and CINERGI - thoughtsScienceBase and CINERGI - thoughts
ScienceBase and CINERGI - thoughtsSky Bristol
 
ScienceBase Architecture - Access Methods
ScienceBase Architecture - Access MethodsScienceBase Architecture - Access Methods
ScienceBase Architecture - Access MethodsSky Bristol
 

More from Sky Bristol (7)

CanyonViews.pptx
CanyonViews.pptxCanyonViews.pptx
CanyonViews.pptx
 
USGS research infrastructure - AGU2016 - in13 e
USGS research infrastructure - AGU2016 - in13 eUSGS research infrastructure - AGU2016 - in13 e
USGS research infrastructure - AGU2016 - in13 e
 
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
 
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
 
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
 
ScienceBase and CINERGI - thoughts
ScienceBase and CINERGI - thoughtsScienceBase and CINERGI - thoughts
ScienceBase and CINERGI - thoughts
 
ScienceBase Architecture - Access Methods
ScienceBase Architecture - Access MethodsScienceBase Architecture - Access Methods
ScienceBase Architecture - Access Methods
 

Recently uploaded

Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio, Inc.
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfPower Karaoke
 
buds n tech IT solutions
buds n  tech IT                solutionsbuds n  tech IT                solutions
buds n tech IT solutionsmonugehlot87
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureDinusha Kumarasiri
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software DevelopersVinodh Ram
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWave PLM
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...MyIntelliSource, Inc.
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningVitsRangannavar
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...kellynguyen01
 
Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Andreas Granig
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideChristina Lin
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...gurkirankumar98700
 
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEBATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEOrtus Solutions, Corp
 
XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsXpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsMehedi Hasan Shohan
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxTier1 app
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfjoe51371421
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...soniya singh
 
What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number SystemsJheuzeDellosa
 
What are the features of Vehicle Tracking System?
What are the features of Vehicle Tracking System?What are the features of Vehicle Tracking System?
What are the features of Vehicle Tracking System?Watsoo Telematics
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...Christina Lin
 

Recently uploaded (20)

Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdf
 
buds n tech IT solutions
buds n  tech IT                solutionsbuds n  tech IT                solutions
buds n tech IT solutions
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with Azure
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software Developers
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need It
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learning
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
 
Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
 
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASEBATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
BATTLEFIELD ORM: TIPS, TACTICS AND STRATEGIES FOR CONQUERING YOUR DATABASE
 
XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsXpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software Solutions
 
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptxKnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
KnowAPIs-UnknownPerf-jaxMainz-2024 (1).pptx
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
 
What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number Systems
 
What are the features of Vehicle Tracking System?
What are the features of Vehicle Tracking System?What are the features of Vehicle Tracking System?
What are the features of Vehicle Tracking System?
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
 

Judicious use of custom development in an open source component architecture

  • 1. Judicious use of custom development in an open source component architecture Sky Bristol1 Natalie Latysh1 Dell Long2 Steve Tekell2 Jeff Allen2 U.S. Department of the Interior U.S. Geological Survey 1. USGS Core Science Analytics, Synthesis and Libraries 2. USGS Fort Collins Science Center
  • 2. AGU Fall Meeting 2014 Talking Points • ScienceBase – inspiration and a brief history • Development methodology • Information architecture • Cyberinfrastructure architecture • API first • Lessons learned and motivators for future • Open invitation (and invite us to your party too)
  • 3. AGU Fall Meeting 2014 myUSGS Data Explorer/Data Uploader Scientific Data Catalog/Compr ehensive Science Catalog ScienceBase 1.0 and then 2.0 ScienceBase- Constellation Collaborative tools and simple file upload Metadata Cataloging & Research Item Concept Digital Repository & Research Item Faceting Federation of Disparate Data Schemas “In the research process, we need more than just a big catalog of data. We need all of the other important information connected to our work – published papers, manuscripts, software, and information about people, labs, projects, and others in our field.” Inspiration and History
  • 4. AGU Fall Meeting 2014 Agile Development
  • 5. AGU Fall Meeting 2014 Flexible Information Architecture
  • 6. AGU Fall Meeting 2014 Cyberinfrastructure Overview
  • 7. AGU Fall Meeting 2014 Manage and Contribute Register Find it at the USGS Find it at the DOI Find it at Data.gov Find it in a Data.gov community (climate.data.gov) Find it in other communities (GeothermalData.org) Open Data and USGIN Methods (“abridged”)
  • 8. AGU Fall Meeting 2014 API First
  • 9. AGU Fall Meeting 2014 Lessons Learned • We encounter some new type of data just about every week and need to account for a lot more variability. • It’s pretty expensive to run an institutional repository, but we have it to do. • Baking security in from the ground up adds quite a few constraints to the ideal architecture. • We can’t please everyone all the time, but if we keep building great platforms then people come back. • APIs (and everything else, for that matter) need to be driven by real, specific use cases to be usable. • R&D, engineering, and operations are all different and need different management methods.
  • 10. AGU Fall Meeting 2014 Community Framework and Platform vs. Agency System
  • 11. AGU Fall Meeting 2014 Questions www.sciencebase.gov sciencebase@usgs.gov Natalie Latysh (nlatysh@usgs.gov) Sky Bristol (sbristol@usgs.gov)