SlideShare a Scribd company logo
1 of 33
Download to read offline
globus online
Science for the Future
Strategies for
distributing and sharing data
www.globusonline.org
Ian Foster
foster@anl.gov
Big science data should be easy
Registry
Staging
Store
Ingest
Store
Analysis
Store
Community
Store
Archive Mirror
Ingest
Store
Analysis
Store
Community
Store
Archive Mirror
Registry
… but it’s hard and frustrating!
Registry
Staging
Store
Ingest
Store
Analysis
Store
Community
Store
Archive Mirror
Ingest
Store
Analysis
Store
Community
Store
Archive Mirror
Registry
Quota
exceeded
!
Expired
credentials
!
Network
failed. Retry.
!
Permission
denied
!
Excerpts from ESNet reports
• “Transfers often take longer than expected
based on available network capacities”
• “Lack of an easy to use interface to some of the
high-performance tools”
• “Tools [are] too difficult to install and use”
• “Time and interruption to other work required to
supervise large data transfers”
• “Need data transfer tools that are easy to use,
well-supported, and permitted by site and facility
cybersecurity organizations”
We envisage a world where data …
… flows rapidly, reliably, and securely
among:
experimental facilities,
online and archival storage,
computing facilities, and
remote institutions
We envisage a world where data …
… is easily integrated into dynamic
datasets that also include metadata
and programs necessary to understand
and regenerate it
We envisage a world where data …
… is readily discoverable and
accessible to collaborators, regardless
of their and the data’s location
We believe a new approach is
needed to deliver data
management infrastructure
Frictionless
Affordable
Sustainable
Like … but for science!
Focusing on “frictionless”, we’ve started to do
this with the Globus Online service …
Transfer and sharing of
large data sets …
… with dropbox-like
characteristics …
… directly from your own
storage systems
We started with reliable, secure,
high-performance file transfer …
Data
Source
Data
Destination
User initiates
transfer request
1
Globus Online
moves and
syncs files
2
Globus Online
notifies user
3
… and then made it simple to share
big data off existing storage systems
Data
Source
User A selects
file(s) to share,
selects user or
group, and sets
permissions
1
Globus Online tracks
shared files; no need
to move files to cloud
storage!
2
User B logs in to
Globus Online and
accesses shared file
3
Early adoption is encouraging
Early adoption is encouraging
~18 PB and 1B files moved
10x (or better) performance vs. scp
99.9% availability
Science for the Future: Strategies for Moving and Sharing Data
B. Winjum (UCLA) moves
900K-file plasma physics
datasets UCLA NERSC
Dan Kozak (Caltech)
replicates 1 PB LIGO
astronomy data for
resilience
Exemplar: APS Beamline 2-BM
X-Ray imaging, tomography, ~few µm to
30nm resolution
Currently can generate
>100TB per day
<1GB/s data rate; ~3-
5GB/s in 5-10 years
Transforming data acquisition
Current
• Experimental parameters
optimized manually
• Collected data combined
with visual inspection to
confirm optimal condition
• Data reconstructed and sent
to users via external drive
• User team starts data
reduction at home institution
Transforming data acquisition
Envisaged
• Experimental parameters
optimized automatically
• Collected data available to
optimization programs
• Data are automatically
reconstructed, reduced,
and shared with local and
remote participants
• User team leaves the APS
with reduced data
Current
• Experimental parameters
optimized manually
• Collected data combined
with visual inspection to
confirm optimal condition
• Data reconstructed and sent
to users via external drive
• User team starts data
reduction at home institution
Facility data
acquisition
Globus Online as enabler
Globus Online
transfer service
Reduced
data
Analysis/Shar
ingGlobus Online
sharing service
Globus Online
dataset service*
* In development
21Credit: Kerstin Kleese-van Dam
Erin Miller (PNNL)
collects data at
Advanced Photon
Source, renders at
PNNL, and views at
ANL
We believe a new approach is
needed to deliver data
management infrastructure
Frictionless
Affordable
Sustainable
We’ve got a handle on “frictionless”
• Web interface, REST API, command line
• InCommon, Oauth, OpenID, X.509, …
• Credential management
• Group definition and management
• Transfer management and optimization
• Reliability via transfer retries
• Integration with ESNet “Science DMZs”
• One-click “Globus Connect” install
• 5-minute Globus Connect Multi User install
“Affordable” and “sustainable”?
Common expectation is either:
– High-priced commercial software
(with generally higher levels of quality)
Or:
– Free, open source software
(with generally lower levels of quality)
We aim to offer the best of all worlds!
We are a non-profit service
provider to the non-profit
research community
Our challenge:
Sustainability
We are a non-profit service
provider to the non-profit
research community
Starting at $20k per year
• Managed endpoints with sharing
• Multiple GridFTP servers per endpoint
• Branded web sites
• Alternate identity provider
• Usage reporting
• Mass storage system (MSS) optimizations
• Operations monitoring and management
• Input into and access to product roadmap
Globus Online Provider Plans
Provider Plan not required to get started
Use Globus Connect Multiuser to easily
connect your resources with Globus
Go to: globusonline.org/gcmu
Registry
Staging
Store
Ingest
Store
Analysis
Store
Community
Store
Archive Mirror
Ingest
Store
Analysis
Store
Community
Store
Archive Mirror
Registry
We hope you will join us
Providers are also using Globus
Online as a platform
Globus Nexus
(Identity, Group, Profile)
…
Sharing Service
Transfer Service
Dataset Services
Globus Toolkit
GlobusOnlineAPIs
GlobusConnect
Early platform adopters
Our research is supported by:
U.S. DE PARTME NT OF
ENERGY
Questions
Contact: support@globusonline.org
Providers: globusonline.org/provider-plans
Researchers: globusonline.org/plus
www.globusonline.org

More Related Content

What's hot

Real Time Business Platform by Ivan Novick from Pivotal
Real Time Business Platform by Ivan Novick from PivotalReal Time Business Platform by Ivan Novick from Pivotal
Real Time Business Platform by Ivan Novick from PivotalVMware Tanzu Korea
 
The future of Hadoop security and its evolution by Alejandro González at Big ...
The future of Hadoop security and its evolution by Alejandro González at Big ...The future of Hadoop security and its evolution by Alejandro González at Big ...
The future of Hadoop security and its evolution by Alejandro González at Big ...Big Data Spain
 
Stream Analytics for Data in Motion
Stream Analytics for Data in MotionStream Analytics for Data in Motion
Stream Analytics for Data in MotionExtraHop Networks
 
Empower your security practitioners with the Elastic Stack
Empower your security practitioners with the Elastic StackEmpower your security practitioners with the Elastic Stack
Empower your security practitioners with the Elastic StackElasticsearch
 
Privacera and Northwestern Mutual - Scaling Privacy in a Spark Ecosystem
Privacera and Northwestern Mutual  - Scaling Privacy in a Spark EcosystemPrivacera and Northwestern Mutual  - Scaling Privacy in a Spark Ecosystem
Privacera and Northwestern Mutual - Scaling Privacy in a Spark EcosystemPrivacera
 
#PCMVision: VMware NSX - Transforming Security
#PCMVision: VMware NSX - Transforming Security#PCMVision: VMware NSX - Transforming Security
#PCMVision: VMware NSX - Transforming SecurityPCM
 
The Elastic Evolution of CenturyLink’s Network Management System
The Elastic Evolution of CenturyLink’s Network Management SystemThe Elastic Evolution of CenturyLink’s Network Management System
The Elastic Evolution of CenturyLink’s Network Management SystemElasticsearch
 
Ech truth init ppt 072213 final
Ech truth init ppt 072213 finalEch truth init ppt 072213 final
Ech truth init ppt 072213 finalEchopath, LLC
 
Redis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured StreamingRedis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured StreamingDave Nielsen
 
Improving Veteran benefit services through efficient data streaming | Robert ...
Improving Veteran benefit services through efficient data streaming | Robert ...Improving Veteran benefit services through efficient data streaming | Robert ...
Improving Veteran benefit services through efficient data streaming | Robert ...HostedbyConfluent
 
Trusted DB a trusted hardware based database with privacy and data confidenti...
Trusted DB a trusted hardware based database with privacy and data confidenti...Trusted DB a trusted hardware based database with privacy and data confidenti...
Trusted DB a trusted hardware based database with privacy and data confidenti...Papitha Velumani
 
Hortonworks Hybrid Cloud - Putting you back in control of your data
Hortonworks Hybrid Cloud - Putting you back in control of your dataHortonworks Hybrid Cloud - Putting you back in control of your data
Hortonworks Hybrid Cloud - Putting you back in control of your dataScott Clinton
 
History of data centric transformation preso
History of data centric transformation presoHistory of data centric transformation preso
History of data centric transformation presoDarren Pulsipher
 
Identity-Based Remote Data Integrity Checking With Perfect Data Privacy Prese...
Identity-Based Remote Data Integrity Checking With Perfect Data Privacy Prese...Identity-Based Remote Data Integrity Checking With Perfect Data Privacy Prese...
Identity-Based Remote Data Integrity Checking With Perfect Data Privacy Prese...Prasadu Peddi
 
Secure Networking in Big Data Environments
Secure Networking in Big Data EnvironmentsSecure Networking in Big Data Environments
Secure Networking in Big Data EnvironmentsNapier University
 
Simplified Research Data Management with the Globus Platform
Simplified Research Data Management with the Globus PlatformSimplified Research Data Management with the Globus Platform
Simplified Research Data Management with the Globus PlatformGlobus
 
Industrial production process visualization with the Elastic Stack in real-ti...
Industrial production process visualization with the Elastic Stack in real-ti...Industrial production process visualization with the Elastic Stack in real-ti...
Industrial production process visualization with the Elastic Stack in real-ti...Elasticsearch
 
Trusted db a trusted hardware based database with privacy and data confidenti...
Trusted db a trusted hardware based database with privacy and data confidenti...Trusted db a trusted hardware based database with privacy and data confidenti...
Trusted db a trusted hardware based database with privacy and data confidenti...LeMeniz Infotech
 
Top 5 Reasons to Replace Data Domain with NetBackup Appliances
Top 5 Reasons to Replace Data Domain with NetBackup AppliancesTop 5 Reasons to Replace Data Domain with NetBackup Appliances
Top 5 Reasons to Replace Data Domain with NetBackup AppliancesVeritas Technologies LLC
 

What's hot (20)

Real Time Business Platform by Ivan Novick from Pivotal
Real Time Business Platform by Ivan Novick from PivotalReal Time Business Platform by Ivan Novick from Pivotal
Real Time Business Platform by Ivan Novick from Pivotal
 
The future of Hadoop security and its evolution by Alejandro González at Big ...
The future of Hadoop security and its evolution by Alejandro González at Big ...The future of Hadoop security and its evolution by Alejandro González at Big ...
The future of Hadoop security and its evolution by Alejandro González at Big ...
 
Stream Analytics for Data in Motion
Stream Analytics for Data in MotionStream Analytics for Data in Motion
Stream Analytics for Data in Motion
 
Empower your security practitioners with the Elastic Stack
Empower your security practitioners with the Elastic StackEmpower your security practitioners with the Elastic Stack
Empower your security practitioners with the Elastic Stack
 
Privacera and Northwestern Mutual - Scaling Privacy in a Spark Ecosystem
Privacera and Northwestern Mutual  - Scaling Privacy in a Spark EcosystemPrivacera and Northwestern Mutual  - Scaling Privacy in a Spark Ecosystem
Privacera and Northwestern Mutual - Scaling Privacy in a Spark Ecosystem
 
#PCMVision: VMware NSX - Transforming Security
#PCMVision: VMware NSX - Transforming Security#PCMVision: VMware NSX - Transforming Security
#PCMVision: VMware NSX - Transforming Security
 
The Elastic Evolution of CenturyLink’s Network Management System
The Elastic Evolution of CenturyLink’s Network Management SystemThe Elastic Evolution of CenturyLink’s Network Management System
The Elastic Evolution of CenturyLink’s Network Management System
 
Ech truth init ppt 072213 final
Ech truth init ppt 072213 finalEch truth init ppt 072213 final
Ech truth init ppt 072213 final
 
Redis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured StreamingRedis Streams plus Spark Structured Streaming
Redis Streams plus Spark Structured Streaming
 
Improving Veteran benefit services through efficient data streaming | Robert ...
Improving Veteran benefit services through efficient data streaming | Robert ...Improving Veteran benefit services through efficient data streaming | Robert ...
Improving Veteran benefit services through efficient data streaming | Robert ...
 
Trusted DB a trusted hardware based database with privacy and data confidenti...
Trusted DB a trusted hardware based database with privacy and data confidenti...Trusted DB a trusted hardware based database with privacy and data confidenti...
Trusted DB a trusted hardware based database with privacy and data confidenti...
 
Hortonworks Hybrid Cloud - Putting you back in control of your data
Hortonworks Hybrid Cloud - Putting you back in control of your dataHortonworks Hybrid Cloud - Putting you back in control of your data
Hortonworks Hybrid Cloud - Putting you back in control of your data
 
History of data centric transformation preso
History of data centric transformation presoHistory of data centric transformation preso
History of data centric transformation preso
 
Leading the Change
Leading the ChangeLeading the Change
Leading the Change
 
Identity-Based Remote Data Integrity Checking With Perfect Data Privacy Prese...
Identity-Based Remote Data Integrity Checking With Perfect Data Privacy Prese...Identity-Based Remote Data Integrity Checking With Perfect Data Privacy Prese...
Identity-Based Remote Data Integrity Checking With Perfect Data Privacy Prese...
 
Secure Networking in Big Data Environments
Secure Networking in Big Data EnvironmentsSecure Networking in Big Data Environments
Secure Networking in Big Data Environments
 
Simplified Research Data Management with the Globus Platform
Simplified Research Data Management with the Globus PlatformSimplified Research Data Management with the Globus Platform
Simplified Research Data Management with the Globus Platform
 
Industrial production process visualization with the Elastic Stack in real-ti...
Industrial production process visualization with the Elastic Stack in real-ti...Industrial production process visualization with the Elastic Stack in real-ti...
Industrial production process visualization with the Elastic Stack in real-ti...
 
Trusted db a trusted hardware based database with privacy and data confidenti...
Trusted db a trusted hardware based database with privacy and data confidenti...Trusted db a trusted hardware based database with privacy and data confidenti...
Trusted db a trusted hardware based database with privacy and data confidenti...
 
Top 5 Reasons to Replace Data Domain with NetBackup Appliances
Top 5 Reasons to Replace Data Domain with NetBackup AppliancesTop 5 Reasons to Replace Data Domain with NetBackup Appliances
Top 5 Reasons to Replace Data Domain with NetBackup Appliances
 

Viewers also liked

Computing Outside The Box September 2009
Computing Outside The Box September 2009Computing Outside The Box September 2009
Computing Outside The Box September 2009Ian Foster
 
Agents In An Exponential World Foster
Agents In An Exponential World FosterAgents In An Exponential World Foster
Agents In An Exponential World FosterIan Foster
 
Recruiting in a Networked World - Workshop Series
Recruiting in a Networked World - Workshop SeriesRecruiting in a Networked World - Workshop Series
Recruiting in a Networked World - Workshop Serieshholmes75
 
Grid And Healthcare For IOM July 2009
Grid And Healthcare For IOM July 2009Grid And Healthcare For IOM July 2009
Grid And Healthcare For IOM July 2009Ian Foster
 
Recruitment and Selection
Recruitment and SelectionRecruitment and Selection
Recruitment and Selectionr m
 
Rethinking how we provide science IT in an era of massive data but modest bud...
Rethinking how we provide science IT in an era of massive data but modest bud...Rethinking how we provide science IT in an era of massive data but modest bud...
Rethinking how we provide science IT in an era of massive data but modest bud...Ian Foster
 
Going farther together - Why Communities are essential for the future of science
Going farther together - Why Communities are essential for the future of scienceGoing farther together - Why Communities are essential for the future of science
Going farther together - Why Communities are essential for the future of scienceErin Robinson
 
Shape of Sound: Sound is Energy.waves.hMa
Shape of Sound: Sound is Energy.waves.hMaShape of Sound: Sound is Energy.waves.hMa
Shape of Sound: Sound is Energy.waves.hMavictoria meyers
 
30.1 oceania landforms and resources
30.1   oceania landforms and resources30.1   oceania landforms and resources
30.1 oceania landforms and resourcesJohn Hext
 
Industrialization And Reform 1820 1850s
Industrialization And Reform 1820 1850sIndustrialization And Reform 1820 1850s
Industrialization And Reform 1820 1850sR. Scudder
 
The Future of Science & Medicine Technology
The Future of Science & Medicine TechnologyThe Future of Science & Medicine Technology
The Future of Science & Medicine TechnologyCity of Boise
 
Chapter 13 Slide Show North And South
Chapter 13 Slide Show North And SouthChapter 13 Slide Show North And South
Chapter 13 Slide Show North And Southlori cuevas
 
Globus Auth: A Research Identity and Access Management Platform
Globus Auth: A Research Identity and Access Management PlatformGlobus Auth: A Research Identity and Access Management Platform
Globus Auth: A Research Identity and Access Management PlatformIan Foster
 
Streamlined data sharing and analysis to accelerate cancer research
Streamlined data sharing and analysis to accelerate cancer researchStreamlined data sharing and analysis to accelerate cancer research
Streamlined data sharing and analysis to accelerate cancer researchIan Foster
 
Road to Civil War PowerPoint
Road to Civil War PowerPointRoad to Civil War PowerPoint
Road to Civil War PowerPointKristin Prusinski
 

Viewers also liked (20)

Computing Outside The Box September 2009
Computing Outside The Box September 2009Computing Outside The Box September 2009
Computing Outside The Box September 2009
 
Agents In An Exponential World Foster
Agents In An Exponential World FosterAgents In An Exponential World Foster
Agents In An Exponential World Foster
 
Recruiting in a Networked World - Workshop Series
Recruiting in a Networked World - Workshop SeriesRecruiting in a Networked World - Workshop Series
Recruiting in a Networked World - Workshop Series
 
Grid And Healthcare For IOM July 2009
Grid And Healthcare For IOM July 2009Grid And Healthcare For IOM July 2009
Grid And Healthcare For IOM July 2009
 
Recruitment and Selection
Recruitment and SelectionRecruitment and Selection
Recruitment and Selection
 
Rethinking how we provide science IT in an era of massive data but modest bud...
Rethinking how we provide science IT in an era of massive data but modest bud...Rethinking how we provide science IT in an era of massive data but modest bud...
Rethinking how we provide science IT in an era of massive data but modest bud...
 
Brodnax 2
Brodnax 2Brodnax 2
Brodnax 2
 
Going farther together - Why Communities are essential for the future of science
Going farther together - Why Communities are essential for the future of scienceGoing farther together - Why Communities are essential for the future of science
Going farther together - Why Communities are essential for the future of science
 
Shape of Sound: Sound is Energy.waves.hMa
Shape of Sound: Sound is Energy.waves.hMaShape of Sound: Sound is Energy.waves.hMa
Shape of Sound: Sound is Energy.waves.hMa
 
30.1 oceania landforms and resources
30.1   oceania landforms and resources30.1   oceania landforms and resources
30.1 oceania landforms and resources
 
Industrialization And Reform 1820 1850s
Industrialization And Reform 1820 1850sIndustrialization And Reform 1820 1850s
Industrialization And Reform 1820 1850s
 
The Future of Science & Medicine Technology
The Future of Science & Medicine TechnologyThe Future of Science & Medicine Technology
The Future of Science & Medicine Technology
 
Chapter 13 Slide Show North And South
Chapter 13 Slide Show North And SouthChapter 13 Slide Show North And South
Chapter 13 Slide Show North And South
 
Globus Auth: A Research Identity and Access Management Platform
Globus Auth: A Research Identity and Access Management PlatformGlobus Auth: A Research Identity and Access Management Platform
Globus Auth: A Research Identity and Access Management Platform
 
Streamlined data sharing and analysis to accelerate cancer research
Streamlined data sharing and analysis to accelerate cancer researchStreamlined data sharing and analysis to accelerate cancer research
Streamlined data sharing and analysis to accelerate cancer research
 
Presentation on world
Presentation on worldPresentation on world
Presentation on world
 
Mapping the earth
Mapping the earthMapping the earth
Mapping the earth
 
Road to Civil War PowerPoint
Road to Civil War PowerPointRoad to Civil War PowerPoint
Road to Civil War PowerPoint
 
US History Chapter 15
US History Chapter 15US History Chapter 15
US History Chapter 15
 
Oceanía
Oceanía Oceanía
Oceanía
 

Similar to Science for the Future: Strategies for Moving and Sharing Data

Science as a Service: How On-Demand Computing can Accelerate Discovery
Science as a Service: How On-Demand Computing can Accelerate DiscoveryScience as a Service: How On-Demand Computing can Accelerate Discovery
Science as a Service: How On-Demand Computing can Accelerate DiscoveryIan Foster
 
Science cloud foster june 2013
Science cloud foster june 2013Science cloud foster june 2013
Science cloud foster june 2013Kirill Osipov
 
Globus status and publication plans
Globus status and publication plansGlobus status and publication plans
Globus status and publication plansIan Foster
 
Introduction to Globus - XSEDE14 Tutorial
Introduction to Globus - XSEDE14 TutorialIntroduction to Globus - XSEDE14 Tutorial
Introduction to Globus - XSEDE14 TutorialGlobus
 
Delivering a Campus Research Data Service with Globus
Delivering a Campus Research Data Service with GlobusDelivering a Campus Research Data Service with Globus
Delivering a Campus Research Data Service with GlobusIan Foster
 
re:Invent 2013-foster-madduri
re:Invent 2013-foster-maddurire:Invent 2013-foster-madduri
re:Invent 2013-foster-madduriRavi Madduri
 
Big Process for Big Data @ NASA
Big Process for Big Data @ NASABig Process for Big Data @ NASA
Big Process for Big Data @ NASAIan Foster
 
Automating Research Data Management at Scale with Globus
Automating Research Data Management at Scale with GlobusAutomating Research Data Management at Scale with Globus
Automating Research Data Management at Scale with GlobusGlobus
 
Webinar: Q&A on Globus Subscription Features
Webinar: Q&A on Globus Subscription FeaturesWebinar: Q&A on Globus Subscription Features
Webinar: Q&A on Globus Subscription FeaturesGlobus
 
Globus Genomics: How Science-as-a-Service is Accelerating Discovery (BDT310) ...
Globus Genomics: How Science-as-a-Service is Accelerating Discovery (BDT310) ...Globus Genomics: How Science-as-a-Service is Accelerating Discovery (BDT310) ...
Globus Genomics: How Science-as-a-Service is Accelerating Discovery (BDT310) ...Amazon Web Services
 
NIH NCI Childhood Cancer Data Initiative (CCDI) Symposium Globus Poster
NIH NCI Childhood Cancer Data Initiative (CCDI) Symposium Globus PosterNIH NCI Childhood Cancer Data Initiative (CCDI) Symposium Globus Poster
NIH NCI Childhood Cancer Data Initiative (CCDI) Symposium Globus PosterGlobus
 
Introduction to Globus
Introduction to GlobusIntroduction to Globus
Introduction to GlobusGlobus
 
Globus for Data Management: 2014 Joint Facility User Forum
Globus for Data Management: 2014 Joint Facility User ForumGlobus for Data Management: 2014 Joint Facility User Forum
Globus for Data Management: 2014 Joint Facility User ForumGlobus
 
(SEC313) Security & Compliance at the Petabyte Scale
(SEC313) Security & Compliance at the Petabyte Scale(SEC313) Security & Compliance at the Petabyte Scale
(SEC313) Security & Compliance at the Petabyte ScaleAmazon Web Services
 
Introduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for ResearchersIntroduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for ResearchersGlobus
 
How To Build A Stable And Robust Base For a “Cloud”
How To Build A Stable And Robust Base For a “Cloud”How To Build A Stable And Robust Base For a “Cloud”
How To Build A Stable And Robust Base For a “Cloud”Hardway Hou
 
GlobusWorld 2019 Opening Keynote
GlobusWorld 2019 Opening KeynoteGlobusWorld 2019 Opening Keynote
GlobusWorld 2019 Opening KeynoteGlobus
 
Climb stateoftheartintro
Climb stateoftheartintroClimb stateoftheartintro
Climb stateoftheartintrothomasrconnor
 

Similar to Science for the Future: Strategies for Moving and Sharing Data (20)

Science as a Service: How On-Demand Computing can Accelerate Discovery
Science as a Service: How On-Demand Computing can Accelerate DiscoveryScience as a Service: How On-Demand Computing can Accelerate Discovery
Science as a Service: How On-Demand Computing can Accelerate Discovery
 
Science cloud foster june 2013
Science cloud foster june 2013Science cloud foster june 2013
Science cloud foster june 2013
 
Globus status and publication plans
Globus status and publication plansGlobus status and publication plans
Globus status and publication plans
 
Introduction to Globus - XSEDE14 Tutorial
Introduction to Globus - XSEDE14 TutorialIntroduction to Globus - XSEDE14 Tutorial
Introduction to Globus - XSEDE14 Tutorial
 
Delivering a Campus Research Data Service with Globus
Delivering a Campus Research Data Service with GlobusDelivering a Campus Research Data Service with Globus
Delivering a Campus Research Data Service with Globus
 
re:Invent 2013-foster-madduri
re:Invent 2013-foster-maddurire:Invent 2013-foster-madduri
re:Invent 2013-foster-madduri
 
Big Process for Big Data @ NASA
Big Process for Big Data @ NASABig Process for Big Data @ NASA
Big Process for Big Data @ NASA
 
Automating Research Data Management at Scale with Globus
Automating Research Data Management at Scale with GlobusAutomating Research Data Management at Scale with Globus
Automating Research Data Management at Scale with Globus
 
Webinar: Q&A on Globus Subscription Features
Webinar: Q&A on Globus Subscription FeaturesWebinar: Q&A on Globus Subscription Features
Webinar: Q&A on Globus Subscription Features
 
Globus Genomics: How Science-as-a-Service is Accelerating Discovery (BDT310) ...
Globus Genomics: How Science-as-a-Service is Accelerating Discovery (BDT310) ...Globus Genomics: How Science-as-a-Service is Accelerating Discovery (BDT310) ...
Globus Genomics: How Science-as-a-Service is Accelerating Discovery (BDT310) ...
 
NIH NCI Childhood Cancer Data Initiative (CCDI) Symposium Globus Poster
NIH NCI Childhood Cancer Data Initiative (CCDI) Symposium Globus PosterNIH NCI Childhood Cancer Data Initiative (CCDI) Symposium Globus Poster
NIH NCI Childhood Cancer Data Initiative (CCDI) Symposium Globus Poster
 
Introduction to Globus
Introduction to GlobusIntroduction to Globus
Introduction to Globus
 
Sept 24 NISO Virtual Conference: Library Data in the Cloud
Sept 24 NISO Virtual Conference: Library Data in the CloudSept 24 NISO Virtual Conference: Library Data in the Cloud
Sept 24 NISO Virtual Conference: Library Data in the Cloud
 
Aspera @ BioData World 18
Aspera @  BioData World 18Aspera @  BioData World 18
Aspera @ BioData World 18
 
Globus for Data Management: 2014 Joint Facility User Forum
Globus for Data Management: 2014 Joint Facility User ForumGlobus for Data Management: 2014 Joint Facility User Forum
Globus for Data Management: 2014 Joint Facility User Forum
 
(SEC313) Security & Compliance at the Petabyte Scale
(SEC313) Security & Compliance at the Petabyte Scale(SEC313) Security & Compliance at the Petabyte Scale
(SEC313) Security & Compliance at the Petabyte Scale
 
Introduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for ResearchersIntroduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for Researchers
 
How To Build A Stable And Robust Base For a “Cloud”
How To Build A Stable And Robust Base For a “Cloud”How To Build A Stable And Robust Base For a “Cloud”
How To Build A Stable And Robust Base For a “Cloud”
 
GlobusWorld 2019 Opening Keynote
GlobusWorld 2019 Opening KeynoteGlobusWorld 2019 Opening Keynote
GlobusWorld 2019 Opening Keynote
 
Climb stateoftheartintro
Climb stateoftheartintroClimb stateoftheartintro
Climb stateoftheartintro
 

More from Ian Foster

Global Services for Global Science March 2023.pptx
Global Services for Global Science March 2023.pptxGlobal Services for Global Science March 2023.pptx
Global Services for Global Science March 2023.pptxIan Foster
 
The Earth System Grid Federation: Origins, Current State, Evolution
The Earth System Grid Federation: Origins, Current State, EvolutionThe Earth System Grid Federation: Origins, Current State, Evolution
The Earth System Grid Federation: Origins, Current State, EvolutionIan Foster
 
Better Information Faster: Programming the Continuum
Better Information Faster: Programming the ContinuumBetter Information Faster: Programming the Continuum
Better Information Faster: Programming the ContinuumIan Foster
 
ESnet6 and Smart Instruments
ESnet6 and Smart InstrumentsESnet6 and Smart Instruments
ESnet6 and Smart InstrumentsIan Foster
 
Linking Scientific Instruments and Computation
Linking Scientific Instruments and ComputationLinking Scientific Instruments and Computation
Linking Scientific Instruments and ComputationIan Foster
 
A Global Research Data Platform: How Globus Services Enable Scientific Discovery
A Global Research Data Platform: How Globus Services Enable Scientific DiscoveryA Global Research Data Platform: How Globus Services Enable Scientific Discovery
A Global Research Data Platform: How Globus Services Enable Scientific DiscoveryIan Foster
 
Foster CRA March 2022.pptx
Foster CRA March 2022.pptxFoster CRA March 2022.pptx
Foster CRA March 2022.pptxIan Foster
 
Big Data, Big Computing, AI, and Environmental Science
Big Data, Big Computing, AI, and Environmental ScienceBig Data, Big Computing, AI, and Environmental Science
Big Data, Big Computing, AI, and Environmental ScienceIan Foster
 
AI at Scale for Materials and Chemistry
AI at Scale for Materials and ChemistryAI at Scale for Materials and Chemistry
AI at Scale for Materials and ChemistryIan Foster
 
Coding the Continuum
Coding the ContinuumCoding the Continuum
Coding the ContinuumIan Foster
 
Data Tribology: Overcoming Data Friction with Cloud Automation
Data Tribology: Overcoming Data Friction with Cloud AutomationData Tribology: Overcoming Data Friction with Cloud Automation
Data Tribology: Overcoming Data Friction with Cloud AutomationIan Foster
 
Research Automation for Data-Driven Discovery
Research Automation for Data-Driven DiscoveryResearch Automation for Data-Driven Discovery
Research Automation for Data-Driven DiscoveryIan Foster
 
Scaling collaborative data science with Globus and Jupyter
Scaling collaborative data science with Globus and JupyterScaling collaborative data science with Globus and Jupyter
Scaling collaborative data science with Globus and JupyterIan Foster
 
Learning Systems for Science
Learning Systems for ScienceLearning Systems for Science
Learning Systems for ScienceIan Foster
 
Data Automation at Light Sources
Data Automation at Light SourcesData Automation at Light Sources
Data Automation at Light SourcesIan Foster
 
Team Argon Summary
Team Argon SummaryTeam Argon Summary
Team Argon SummaryIan Foster
 
Thoughts on interoperability
Thoughts on interoperabilityThoughts on interoperability
Thoughts on interoperabilityIan Foster
 
Computing Just What You Need: Online Data Analysis and Reduction at Extreme ...
Computing Just What You Need: Online Data Analysis and Reduction  at Extreme ...Computing Just What You Need: Online Data Analysis and Reduction  at Extreme ...
Computing Just What You Need: Online Data Analysis and Reduction at Extreme ...Ian Foster
 
NIH Data Commons Architecture Ideas
NIH Data Commons Architecture IdeasNIH Data Commons Architecture Ideas
NIH Data Commons Architecture IdeasIan Foster
 
Going Smart and Deep on Materials at ALCF
Going Smart and Deep on Materials at ALCFGoing Smart and Deep on Materials at ALCF
Going Smart and Deep on Materials at ALCFIan Foster
 

More from Ian Foster (20)

Global Services for Global Science March 2023.pptx
Global Services for Global Science March 2023.pptxGlobal Services for Global Science March 2023.pptx
Global Services for Global Science March 2023.pptx
 
The Earth System Grid Federation: Origins, Current State, Evolution
The Earth System Grid Federation: Origins, Current State, EvolutionThe Earth System Grid Federation: Origins, Current State, Evolution
The Earth System Grid Federation: Origins, Current State, Evolution
 
Better Information Faster: Programming the Continuum
Better Information Faster: Programming the ContinuumBetter Information Faster: Programming the Continuum
Better Information Faster: Programming the Continuum
 
ESnet6 and Smart Instruments
ESnet6 and Smart InstrumentsESnet6 and Smart Instruments
ESnet6 and Smart Instruments
 
Linking Scientific Instruments and Computation
Linking Scientific Instruments and ComputationLinking Scientific Instruments and Computation
Linking Scientific Instruments and Computation
 
A Global Research Data Platform: How Globus Services Enable Scientific Discovery
A Global Research Data Platform: How Globus Services Enable Scientific DiscoveryA Global Research Data Platform: How Globus Services Enable Scientific Discovery
A Global Research Data Platform: How Globus Services Enable Scientific Discovery
 
Foster CRA March 2022.pptx
Foster CRA March 2022.pptxFoster CRA March 2022.pptx
Foster CRA March 2022.pptx
 
Big Data, Big Computing, AI, and Environmental Science
Big Data, Big Computing, AI, and Environmental ScienceBig Data, Big Computing, AI, and Environmental Science
Big Data, Big Computing, AI, and Environmental Science
 
AI at Scale for Materials and Chemistry
AI at Scale for Materials and ChemistryAI at Scale for Materials and Chemistry
AI at Scale for Materials and Chemistry
 
Coding the Continuum
Coding the ContinuumCoding the Continuum
Coding the Continuum
 
Data Tribology: Overcoming Data Friction with Cloud Automation
Data Tribology: Overcoming Data Friction with Cloud AutomationData Tribology: Overcoming Data Friction with Cloud Automation
Data Tribology: Overcoming Data Friction with Cloud Automation
 
Research Automation for Data-Driven Discovery
Research Automation for Data-Driven DiscoveryResearch Automation for Data-Driven Discovery
Research Automation for Data-Driven Discovery
 
Scaling collaborative data science with Globus and Jupyter
Scaling collaborative data science with Globus and JupyterScaling collaborative data science with Globus and Jupyter
Scaling collaborative data science with Globus and Jupyter
 
Learning Systems for Science
Learning Systems for ScienceLearning Systems for Science
Learning Systems for Science
 
Data Automation at Light Sources
Data Automation at Light SourcesData Automation at Light Sources
Data Automation at Light Sources
 
Team Argon Summary
Team Argon SummaryTeam Argon Summary
Team Argon Summary
 
Thoughts on interoperability
Thoughts on interoperabilityThoughts on interoperability
Thoughts on interoperability
 
Computing Just What You Need: Online Data Analysis and Reduction at Extreme ...
Computing Just What You Need: Online Data Analysis and Reduction  at Extreme ...Computing Just What You Need: Online Data Analysis and Reduction  at Extreme ...
Computing Just What You Need: Online Data Analysis and Reduction at Extreme ...
 
NIH Data Commons Architecture Ideas
NIH Data Commons Architecture IdeasNIH Data Commons Architecture Ideas
NIH Data Commons Architecture Ideas
 
Going Smart and Deep on Materials at ALCF
Going Smart and Deep on Materials at ALCFGoing Smart and Deep on Materials at ALCF
Going Smart and Deep on Materials at ALCF
 

Recently uploaded

Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 

Recently uploaded (20)

Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 

Science for the Future: Strategies for Moving and Sharing Data

  • 1. globus online Science for the Future Strategies for distributing and sharing data www.globusonline.org Ian Foster foster@anl.gov
  • 2. Big science data should be easy Registry Staging Store Ingest Store Analysis Store Community Store Archive Mirror Ingest Store Analysis Store Community Store Archive Mirror Registry
  • 3. … but it’s hard and frustrating! Registry Staging Store Ingest Store Analysis Store Community Store Archive Mirror Ingest Store Analysis Store Community Store Archive Mirror Registry Quota exceeded ! Expired credentials ! Network failed. Retry. ! Permission denied !
  • 4. Excerpts from ESNet reports • “Transfers often take longer than expected based on available network capacities” • “Lack of an easy to use interface to some of the high-performance tools” • “Tools [are] too difficult to install and use” • “Time and interruption to other work required to supervise large data transfers” • “Need data transfer tools that are easy to use, well-supported, and permitted by site and facility cybersecurity organizations”
  • 5. We envisage a world where data … … flows rapidly, reliably, and securely among: experimental facilities, online and archival storage, computing facilities, and remote institutions
  • 6. We envisage a world where data … … is easily integrated into dynamic datasets that also include metadata and programs necessary to understand and regenerate it
  • 7. We envisage a world where data … … is readily discoverable and accessible to collaborators, regardless of their and the data’s location
  • 8. We believe a new approach is needed to deliver data management infrastructure Frictionless Affordable Sustainable Like … but for science!
  • 9. Focusing on “frictionless”, we’ve started to do this with the Globus Online service … Transfer and sharing of large data sets … … with dropbox-like characteristics … … directly from your own storage systems
  • 10. We started with reliable, secure, high-performance file transfer … Data Source Data Destination User initiates transfer request 1 Globus Online moves and syncs files 2 Globus Online notifies user 3
  • 11. … and then made it simple to share big data off existing storage systems Data Source User A selects file(s) to share, selects user or group, and sets permissions 1 Globus Online tracks shared files; no need to move files to cloud storage! 2 User B logs in to Globus Online and accesses shared file 3
  • 12. Early adoption is encouraging
  • 13. Early adoption is encouraging ~18 PB and 1B files moved 10x (or better) performance vs. scp 99.9% availability
  • 15. B. Winjum (UCLA) moves 900K-file plasma physics datasets UCLA NERSC
  • 16. Dan Kozak (Caltech) replicates 1 PB LIGO astronomy data for resilience
  • 17. Exemplar: APS Beamline 2-BM X-Ray imaging, tomography, ~few µm to 30nm resolution Currently can generate >100TB per day <1GB/s data rate; ~3- 5GB/s in 5-10 years
  • 18. Transforming data acquisition Current • Experimental parameters optimized manually • Collected data combined with visual inspection to confirm optimal condition • Data reconstructed and sent to users via external drive • User team starts data reduction at home institution
  • 19. Transforming data acquisition Envisaged • Experimental parameters optimized automatically • Collected data available to optimization programs • Data are automatically reconstructed, reduced, and shared with local and remote participants • User team leaves the APS with reduced data Current • Experimental parameters optimized manually • Collected data combined with visual inspection to confirm optimal condition • Data reconstructed and sent to users via external drive • User team starts data reduction at home institution
  • 20. Facility data acquisition Globus Online as enabler Globus Online transfer service Reduced data Analysis/Shar ingGlobus Online sharing service Globus Online dataset service* * In development
  • 21. 21Credit: Kerstin Kleese-van Dam Erin Miller (PNNL) collects data at Advanced Photon Source, renders at PNNL, and views at ANL
  • 22. We believe a new approach is needed to deliver data management infrastructure Frictionless Affordable Sustainable
  • 23. We’ve got a handle on “frictionless” • Web interface, REST API, command line • InCommon, Oauth, OpenID, X.509, … • Credential management • Group definition and management • Transfer management and optimization • Reliability via transfer retries • Integration with ESNet “Science DMZs” • One-click “Globus Connect” install • 5-minute Globus Connect Multi User install
  • 24. “Affordable” and “sustainable”? Common expectation is either: – High-priced commercial software (with generally higher levels of quality) Or: – Free, open source software (with generally lower levels of quality) We aim to offer the best of all worlds!
  • 25. We are a non-profit service provider to the non-profit research community
  • 26. Our challenge: Sustainability We are a non-profit service provider to the non-profit research community
  • 27. Starting at $20k per year • Managed endpoints with sharing • Multiple GridFTP servers per endpoint • Branded web sites • Alternate identity provider • Usage reporting • Mass storage system (MSS) optimizations • Operations monitoring and management • Input into and access to product roadmap Globus Online Provider Plans
  • 28. Provider Plan not required to get started Use Globus Connect Multiuser to easily connect your resources with Globus Go to: globusonline.org/gcmu Registry Staging Store Ingest Store Analysis Store Community Store Archive Mirror Ingest Store Analysis Store Community Store Archive Mirror Registry
  • 29. We hope you will join us
  • 30. Providers are also using Globus Online as a platform Globus Nexus (Identity, Group, Profile) … Sharing Service Transfer Service Dataset Services Globus Toolkit GlobusOnlineAPIs GlobusConnect
  • 32. Our research is supported by: U.S. DE PARTME NT OF ENERGY

Editor's Notes

  1. This image shows a 3D rendering of a Shewanella biofilm grown on a flat plastic substrate in a Constant Depth bioFilm Fermenter (CDFF).  The image was generated using x-ray microtomography at the Advanced Photon Source, Argonne National Laboratory.   
  2. Mention plans start from 20K and that we are talking to NET+ for campus plans with separate pricing?