SlideShare a Scribd company logo
1 of 26
Download to read offline
Rachana Ananthakrishnan
ranantha@uchicago.edu
February 27, 2024
Introduction to Globus for Researchers
Globus is …
a non-profit service
developed and operated by
Globus Platform for Research IT
Managed transfer & sync
Collaborative data sharing
Unified data access
Publication & discovery
Reliable automation Platform-as-a-Service
Managed remote execution
Software-as-a-Service
One service, many interfaces
4
GET /endpoint/go%23ep1
PUT /endpoint/vas#my_endpt
200 OK
X-Transfer-API-Version: 0.10
Content-Type: application/json
…
Globus service
Web
CLI
Rest
API
Flows
Fast, reliable file transfer …from any to any system
User-initiated,
or automated
transfer request
1
Instrument,
Lab server
Compute
Facility
Globus transfers files
reliably, securely
2
Globally accessible
multi-tenant service
• Fire-and-forget
transfers/sync
• Optimized speed
• Assured reliability
• Unified view of storage
• HTTP/S access to data
v
Optional
notifications
3
Globus is reliable, secure …and fast!
72.8Gbps
7
Misconfigured
GPFS
ALCF
maintenance
LLNLà
ALCF
LLNLà
OLCF
ALCFà
OLCF
OLCFà
ALCF
17,347,671 directories and 28,907,532 files
February 12 to May 4, 2022
Sustained rate 1.45 GB/s (LLNL rate limit)
Peak 7.5 GB/s (OLCFàALCF)
Challenge: Replicate 7.5 PB of
climate data to Argonne, Oak Ridge
Connecting your
storage system
8
Endpoints, Collections and
Globus Connect
• Globus Connect Server
– for multi-user Linux Systems
docs.globus.org/globus-connect-server
• Globus Connect Personal
– for personal workstations and laptops
globus.org/globus-connect-personal
docs.globus.org/how-to
Collections for data access
• Directly addressable entities
• Bulk data access (via Globus transfer service)
• HTTP/S access directly from collection
• Connected to a storage system, and policy managed
by institution
Mapped Collections
Preparing for the
exercises
bit.ly/gw-tut-rpi
11
Let’s try it…
• Discover collections
• Data access
– Transfer and sync
– Upload/download
• Transferring from laptops/desktops
– GCP install
Globus supports diverse storage systems
Move without (worrying about) limits
• API request rates
• File size
• Data volume
• Third-party tools cannot circumvent…
• …but Globus lets you “fire-and-forget”
• à it will (eventually) be done
14
Uniform interface, consistent user experience
15
Best practices for data transfer
• Submit all data in single task
– Smaller number of large tasks for best performance
• Choose sync options carefully
– Checksum sync has the most overhead
• Filters are applied separately for
– Listing
– Transfer
Secure data sharing …from any storage
Collaborator logs into Globus
and accesses shared files;
no local account required;
download via Globus
2
On-prem or
public cloud
storage
Select files to share,
select user or group,
and set access
permissions
1
Globally accessible
multi-tenant service
Globus controls
access to shared files
on existing storage
Laptop, server,
compute facility
• Fine-grained access
control “overlay” on
storage system
• Share with any identity,
email, group
• No need to stage data just
for sharing
v
Collections for data access and sharing
• Directly addressable entities
• Bulk data access (via Globus transfer service)
• HTTP/S access directly from collection
• Connected to a storage system, and policy managed by institution
• Guest collections include collaborative data sharing
Guest Collections
Mapped Collections
Data sharing permissions management
• Permissions are set per folder, on a guest collection
• Permissions management can be automated
• For a user
– Identity: user must log in with this
– Email: user gets a code via email; link to their Globus Account
• For a group
– Group UUID: search for group to get UUID
– Access governed by membership in the group
• For an application
– Application identity: appclientid@clients.auth.globus.org
Let’s try it…
• Discover collections
• Create guest collection
• Set permissions
Tutorial cheatsheet: bit.ly/gw-tut-rpi
Data sharing roles management
• Roles can be used to grant rights to other users,
groups or applications
• Roles for management of guest collection
– Administrator
– Access manager
• Roles for management of activity on guest collection
– Activity Manager
– Activity Monitor
Let’s try it…
• Discover collections
• Create guest collection
• Set permissions
• Set Roles
• Console for management (admin)
Tutorial cheatsheet: bit.ly/gw-tut-rpi
Globus core security features
• Access Control
– Identities provided and managed by institution
– Institution controls all access policies
– Globus is identity broker; no access to/storage of user credentials
• Data remain at institutions, no storage/routing via Globus
• Integrity checks of transferred data
• Enforced encryption of Globus control data
• Institution-configured encryption of user data in transit
Globus High Assurance for managing protected data
Restricted data
handling
à PHI, PII, CUI
à Compliant
data sharing
Security controls
à NIST 800-53
à 800-171 Low+
BAA w/Uchicago
à UChicago BAA with Amazon
Resources
• Globus home: www.globus.org
• Globus documentation: docs.globus.org
• Globus web app: app.globus.org/login
• Support: support@globus.org
• Education: outreach@globus.org
• Subscriptions: www.globus.org/subscriptions

More Related Content

Similar to An Introduction to Globus for Researchers

Introduction to Globus for Researchers
Introduction to Globus for ResearchersIntroduction to Globus for Researchers
Introduction to Globus for ResearchersGlobus
 
Best Practices for Data Sharing (CHPC 2019 - South Africa)
Best Practices for Data Sharing (CHPC 2019 - South Africa)Best Practices for Data Sharing (CHPC 2019 - South Africa)
Best Practices for Data Sharing (CHPC 2019 - South Africa)Globus
 
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)Best Practices for Data Sharing (GlobusWorld Tour - UCSD)
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)Globus
 
Automating Research Data Management at Scale with Globus
Automating Research Data Management at Scale with GlobusAutomating Research Data Management at Scale with Globus
Automating Research Data Management at Scale with GlobusGlobus
 
Introduction to Globus (GlobusWorld Tour West)
Introduction to Globus (GlobusWorld Tour West)Introduction to Globus (GlobusWorld Tour West)
Introduction to Globus (GlobusWorld Tour West)Globus
 
GlobusWorld 2021 Tutorial: Introduction to Globus
GlobusWorld 2021 Tutorial: Introduction to GlobusGlobusWorld 2021 Tutorial: Introduction to Globus
GlobusWorld 2021 Tutorial: Introduction to GlobusGlobus
 
Globus: Beyond File Transfer
Globus: Beyond File TransferGlobus: Beyond File Transfer
Globus: Beyond File TransferGlobus
 
Introduction to Globus for New Users
Introduction to Globus for New UsersIntroduction to Globus for New Users
Introduction to Globus for New UsersGlobus
 
Introduction to Globus
Introduction to GlobusIntroduction to Globus
Introduction to GlobusGlobus
 
Best Practices for Data Sharing (GlobusWorld Tour - Columbia University)
Best Practices for Data Sharing (GlobusWorld Tour - Columbia University)Best Practices for Data Sharing (GlobusWorld Tour - Columbia University)
Best Practices for Data Sharing (GlobusWorld Tour - Columbia University)Globus
 
Introduction to Globus (APS Workshop)
Introduction to Globus (APS Workshop)Introduction to Globus (APS Workshop)
Introduction to Globus (APS Workshop)Globus
 
Introduction to Globus (GlobusWorld Tour - UMich)
Introduction to Globus (GlobusWorld Tour - UMich)Introduction to Globus (GlobusWorld Tour - UMich)
Introduction to Globus (GlobusWorld Tour - UMich)Globus
 
Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)
Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)
Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)Globus
 
Globus High Assurance for Protected Data (GlobusWorld Tour - Columbia Univers...
Globus High Assurance for Protected Data (GlobusWorld Tour - Columbia Univers...Globus High Assurance for Protected Data (GlobusWorld Tour - Columbia Univers...
Globus High Assurance for Protected Data (GlobusWorld Tour - Columbia Univers...Globus
 
Jupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data ScienceJupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data ScienceGlobus
 
Globus status and publication plans
Globus status and publication plansGlobus status and publication plans
Globus status and publication plansIan Foster
 
Scalable Data Management: Automation and the Modern Research Data Portal
Scalable Data Management: Automation and the Modern Research Data PortalScalable Data Management: Automation and the Modern Research Data Portal
Scalable Data Management: Automation and the Modern Research Data PortalGlobus
 
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)Globus
 

Similar to An Introduction to Globus for Researchers (20)

Introduction to Globus for Researchers
Introduction to Globus for ResearchersIntroduction to Globus for Researchers
Introduction to Globus for Researchers
 
Best Practices for Data Sharing (CHPC 2019 - South Africa)
Best Practices for Data Sharing (CHPC 2019 - South Africa)Best Practices for Data Sharing (CHPC 2019 - South Africa)
Best Practices for Data Sharing (CHPC 2019 - South Africa)
 
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)Best Practices for Data Sharing (GlobusWorld Tour - UCSD)
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)
 
Automating Research Data Management at Scale with Globus
Automating Research Data Management at Scale with GlobusAutomating Research Data Management at Scale with Globus
Automating Research Data Management at Scale with Globus
 
Introduction to Globus (GlobusWorld Tour West)
Introduction to Globus (GlobusWorld Tour West)Introduction to Globus (GlobusWorld Tour West)
Introduction to Globus (GlobusWorld Tour West)
 
GlobusWorld 2021 Tutorial: Introduction to Globus
GlobusWorld 2021 Tutorial: Introduction to GlobusGlobusWorld 2021 Tutorial: Introduction to Globus
GlobusWorld 2021 Tutorial: Introduction to Globus
 
Globus: Beyond File Transfer
Globus: Beyond File TransferGlobus: Beyond File Transfer
Globus: Beyond File Transfer
 
Introduction to Globus for New Users
Introduction to Globus for New UsersIntroduction to Globus for New Users
Introduction to Globus for New Users
 
Introduction to Globus
Introduction to GlobusIntroduction to Globus
Introduction to Globus
 
Globus presentation
Globus presentationGlobus presentation
Globus presentation
 
Best Practices for Data Sharing (GlobusWorld Tour - Columbia University)
Best Practices for Data Sharing (GlobusWorld Tour - Columbia University)Best Practices for Data Sharing (GlobusWorld Tour - Columbia University)
Best Practices for Data Sharing (GlobusWorld Tour - Columbia University)
 
Introduction to Globus (APS Workshop)
Introduction to Globus (APS Workshop)Introduction to Globus (APS Workshop)
Introduction to Globus (APS Workshop)
 
Introduction to Globus (GlobusWorld Tour - UMich)
Introduction to Globus (GlobusWorld Tour - UMich)Introduction to Globus (GlobusWorld Tour - UMich)
Introduction to Globus (GlobusWorld Tour - UMich)
 
Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)
Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)
Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)
 
Globus High Assurance for Protected Data (GlobusWorld Tour - Columbia Univers...
Globus High Assurance for Protected Data (GlobusWorld Tour - Columbia Univers...Globus High Assurance for Protected Data (GlobusWorld Tour - Columbia Univers...
Globus High Assurance for Protected Data (GlobusWorld Tour - Columbia Univers...
 
Jupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data ScienceJupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data Science
 
Globus status and publication plans
Globus status and publication plansGlobus status and publication plans
Globus status and publication plans
 
Sept 24 NISO Virtual Conference: Library Data in the Cloud
Sept 24 NISO Virtual Conference: Library Data in the CloudSept 24 NISO Virtual Conference: Library Data in the Cloud
Sept 24 NISO Virtual Conference: Library Data in the Cloud
 
Scalable Data Management: Automation and the Modern Research Data Portal
Scalable Data Management: Automation and the Modern Research Data PortalScalable Data Management: Automation and the Modern Research Data Portal
Scalable Data Management: Automation and the Modern Research Data Portal
 
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
 

More from Globus

Advanced Globus System Administration Topics
Advanced Globus System Administration TopicsAdvanced Globus System Administration Topics
Advanced Globus System Administration TopicsGlobus
 
Instrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a FlowInstrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a FlowGlobus
 
Building Research Applications with Globus PaaS
Building Research Applications with Globus PaaSBuilding Research Applications with Globus PaaS
Building Research Applications with Globus PaaSGlobus
 
Reliable, Remote Computation at All Scales
Reliable, Remote Computation at All ScalesReliable, Remote Computation at All Scales
Reliable, Remote Computation at All ScalesGlobus
 
Introduction to Research Automation with Globus
Introduction to Research Automation with GlobusIntroduction to Research Automation with Globus
Introduction to Research Automation with GlobusGlobus
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System AdministratorsGlobus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System AdministratorsGlobus
 
Introduction to the Globus Platform for Developers
Introduction to the Globus Platform for DevelopersIntroduction to the Globus Platform for Developers
Introduction to the Globus Platform for DevelopersGlobus
 
Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)Globus
 
Automating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and ComputeAutomating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and ComputeGlobus
 
Automating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus PlatformAutomating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus PlatformGlobus
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System AdministrationGlobus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System AdministratorsGlobus
 
Working with Globus Platform Services and Portals
Working with Globus Platform Services and PortalsWorking with Globus Platform Services and Portals
Working with Globus Platform Services and PortalsGlobus
 
Globus Automation
Globus AutomationGlobus Automation
Globus AutomationGlobus
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System AdministrationGlobus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System AdministratorsGlobus
 
Working with Globus Platform Services
Working with Globus Platform ServicesWorking with Globus Platform Services
Working with Globus Platform ServicesGlobus
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System AdministrationGlobus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System AdministratorsGlobus
 

More from Globus (20)

Advanced Globus System Administration Topics
Advanced Globus System Administration TopicsAdvanced Globus System Administration Topics
Advanced Globus System Administration Topics
 
Instrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a FlowInstrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a Flow
 
Building Research Applications with Globus PaaS
Building Research Applications with Globus PaaSBuilding Research Applications with Globus PaaS
Building Research Applications with Globus PaaS
 
Reliable, Remote Computation at All Scales
Reliable, Remote Computation at All ScalesReliable, Remote Computation at All Scales
Reliable, Remote Computation at All Scales
 
Introduction to Research Automation with Globus
Introduction to Research Automation with GlobusIntroduction to Research Automation with Globus
Introduction to Research Automation with Globus
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 
Introduction to the Globus Platform for Developers
Introduction to the Globus Platform for DevelopersIntroduction to the Globus Platform for Developers
Introduction to the Globus Platform for Developers
 
Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)
 
Automating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and ComputeAutomating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and Compute
 
Automating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus PlatformAutomating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus Platform
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 
Working with Globus Platform Services and Portals
Working with Globus Platform Services and PortalsWorking with Globus Platform Services and Portals
Working with Globus Platform Services and Portals
 
Globus Automation
Globus AutomationGlobus Automation
Globus Automation
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 
Working with Globus Platform Services
Working with Globus Platform ServicesWorking with Globus Platform Services
Working with Globus Platform Services
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 

Recently uploaded

ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBUETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBUsamruddhijedgule2004
 
Key Steps in Agile Software Delivery Roadmap
Key Steps in Agile Software Delivery RoadmapKey Steps in Agile Software Delivery Roadmap
Key Steps in Agile Software Delivery RoadmapIshara Amarasekera
 
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdfSteve Caron
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxAS Design & AST.
 
Zer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfZer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfmaor17
 
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...OnePlan Solutions
 
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...Milind Agarwal
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldRoberto Pérez Alcolea
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdfAndrey Devyatkin
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesVictoriaMetrics
 
oracle 23c new features for developer and dba
oracle 23c new features for developer and dbaoracle 23c new features for developer and dba
oracle 23c new features for developer and dbaRemote DBA Services
 
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive ReviewRevolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Reviewjw364beach
 
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...kalichargn70th171
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jNeo4j
 
Advantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxAdvantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxRTS corp
 
logical backup of Oracle Datapump-detailed.pptx
logical backup of Oracle Datapump-detailed.pptxlogical backup of Oracle Datapump-detailed.pptx
logical backup of Oracle Datapump-detailed.pptxRemote DBA Services
 
OpenMetadata Community Meeting - 4th April, 2024
OpenMetadata Community Meeting - 4th April, 2024OpenMetadata Community Meeting - 4th April, 2024
OpenMetadata Community Meeting - 4th April, 2024OpenMetadata
 
What are the core components of Azure Data Engineer courses.docx
What are the core components of Azure Data Engineer courses.docxWhat are the core components of Azure Data Engineer courses.docx
What are the core components of Azure Data Engineer courses.docxkzayra69
 
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...OnePlan Solutions
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 

Recently uploaded (20)

ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBUETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
ETE PPT.pdf LMMKLMKLMLKMLLMJKBHJBHBNUIHBU
 
Key Steps in Agile Software Delivery Roadmap
Key Steps in Agile Software Delivery RoadmapKey Steps in Agile Software Delivery Roadmap
Key Steps in Agile Software Delivery Roadmap
 
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
 
Mastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptxMastering Project Planning with Microsoft Project 2016.pptx
Mastering Project Planning with Microsoft Project 2016.pptx
 
Zer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfZer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdf
 
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
 
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
Leveraging the Expertise of a Social Media Fraud Analyst to Safeguard Brand R...
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository world
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
oracle 23c new features for developer and dba
oracle 23c new features for developer and dbaoracle 23c new features for developer and dba
oracle 23c new features for developer and dba
 
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive ReviewRevolutionize Your Video Editing with InVideo.io: A Comprehensive Review
Revolutionize Your Video Editing with InVideo.io: A Comprehensive Review
 
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
 
Advantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxAdvantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptx
 
logical backup of Oracle Datapump-detailed.pptx
logical backup of Oracle Datapump-detailed.pptxlogical backup of Oracle Datapump-detailed.pptx
logical backup of Oracle Datapump-detailed.pptx
 
OpenMetadata Community Meeting - 4th April, 2024
OpenMetadata Community Meeting - 4th April, 2024OpenMetadata Community Meeting - 4th April, 2024
OpenMetadata Community Meeting - 4th April, 2024
 
What are the core components of Azure Data Engineer courses.docx
What are the core components of Azure Data Engineer courses.docxWhat are the core components of Azure Data Engineer courses.docx
What are the core components of Azure Data Engineer courses.docx
 
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
Transform your Corporate Strategy Office - Harness OnePlan’s Strategic Portfo...
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 

An Introduction to Globus for Researchers

  • 1. Rachana Ananthakrishnan ranantha@uchicago.edu February 27, 2024 Introduction to Globus for Researchers
  • 2. Globus is … a non-profit service developed and operated by
  • 3. Globus Platform for Research IT Managed transfer & sync Collaborative data sharing Unified data access Publication & discovery Reliable automation Platform-as-a-Service Managed remote execution Software-as-a-Service
  • 4. One service, many interfaces 4 GET /endpoint/go%23ep1 PUT /endpoint/vas#my_endpt 200 OK X-Transfer-API-Version: 0.10 Content-Type: application/json … Globus service Web CLI Rest API Flows
  • 5. Fast, reliable file transfer …from any to any system User-initiated, or automated transfer request 1 Instrument, Lab server Compute Facility Globus transfers files reliably, securely 2 Globally accessible multi-tenant service • Fire-and-forget transfers/sync • Optimized speed • Assured reliability • Unified view of storage • HTTP/S access to data v Optional notifications 3
  • 6. Globus is reliable, secure …and fast! 72.8Gbps
  • 7. 7 Misconfigured GPFS ALCF maintenance LLNLà ALCF LLNLà OLCF ALCFà OLCF OLCFà ALCF 17,347,671 directories and 28,907,532 files February 12 to May 4, 2022 Sustained rate 1.45 GB/s (LLNL rate limit) Peak 7.5 GB/s (OLCFàALCF) Challenge: Replicate 7.5 PB of climate data to Argonne, Oak Ridge
  • 9. Endpoints, Collections and Globus Connect • Globus Connect Server – for multi-user Linux Systems docs.globus.org/globus-connect-server • Globus Connect Personal – for personal workstations and laptops globus.org/globus-connect-personal docs.globus.org/how-to
  • 10. Collections for data access • Directly addressable entities • Bulk data access (via Globus transfer service) • HTTP/S access directly from collection • Connected to a storage system, and policy managed by institution Mapped Collections
  • 12. Let’s try it… • Discover collections • Data access – Transfer and sync – Upload/download • Transferring from laptops/desktops – GCP install
  • 13. Globus supports diverse storage systems
  • 14. Move without (worrying about) limits • API request rates • File size • Data volume • Third-party tools cannot circumvent… • …but Globus lets you “fire-and-forget” • à it will (eventually) be done 14
  • 15. Uniform interface, consistent user experience 15
  • 16. Best practices for data transfer • Submit all data in single task – Smaller number of large tasks for best performance • Choose sync options carefully – Checksum sync has the most overhead • Filters are applied separately for – Listing – Transfer
  • 17. Secure data sharing …from any storage Collaborator logs into Globus and accesses shared files; no local account required; download via Globus 2 On-prem or public cloud storage Select files to share, select user or group, and set access permissions 1 Globally accessible multi-tenant service Globus controls access to shared files on existing storage Laptop, server, compute facility • Fine-grained access control “overlay” on storage system • Share with any identity, email, group • No need to stage data just for sharing v
  • 18. Collections for data access and sharing • Directly addressable entities • Bulk data access (via Globus transfer service) • HTTP/S access directly from collection • Connected to a storage system, and policy managed by institution • Guest collections include collaborative data sharing Guest Collections Mapped Collections
  • 19. Data sharing permissions management • Permissions are set per folder, on a guest collection • Permissions management can be automated • For a user – Identity: user must log in with this – Email: user gets a code via email; link to their Globus Account • For a group – Group UUID: search for group to get UUID – Access governed by membership in the group • For an application – Application identity: appclientid@clients.auth.globus.org
  • 20. Let’s try it… • Discover collections • Create guest collection • Set permissions Tutorial cheatsheet: bit.ly/gw-tut-rpi
  • 21. Data sharing roles management • Roles can be used to grant rights to other users, groups or applications • Roles for management of guest collection – Administrator – Access manager • Roles for management of activity on guest collection – Activity Manager – Activity Monitor
  • 22. Let’s try it… • Discover collections • Create guest collection • Set permissions • Set Roles • Console for management (admin) Tutorial cheatsheet: bit.ly/gw-tut-rpi
  • 23. Globus core security features • Access Control – Identities provided and managed by institution – Institution controls all access policies – Globus is identity broker; no access to/storage of user credentials • Data remain at institutions, no storage/routing via Globus • Integrity checks of transferred data • Enforced encryption of Globus control data • Institution-configured encryption of user data in transit
  • 24. Globus High Assurance for managing protected data Restricted data handling à PHI, PII, CUI à Compliant data sharing Security controls à NIST 800-53 à 800-171 Low+ BAA w/Uchicago à UChicago BAA with Amazon
  • 25.
  • 26. Resources • Globus home: www.globus.org • Globus documentation: docs.globus.org • Globus web app: app.globus.org/login • Support: support@globus.org • Education: outreach@globus.org • Subscriptions: www.globus.org/subscriptions