Introduction to the Globus Platform (GlobusWorld Tour - UMich)

Globus
Introduction to the Globus Platform
Greg Nawrocki
nawrocki@uchicago.edu
greg@globus.org
University of Michigan
July 23, 2019
cloud4scieng.org
Industry software builds on platform services
3
Globus delivers… with applications and as a
platform…
Fast and reliable data transfer, sharing, and file
management…
…directly from your own storage systems…
...via software-as-a-service using existing
identities.
How can I integrate
Globus into my research
workflows?
4
Globus serves as…
A platform for building science
gateways, portals and other
web applications in support of
research and education.
5
Globus Auth API
(Group Management)
…
GlobusTransferAPI
GlobusConnect
File Sharing
File Transfer & Replication
Globus Platform-as-a-Service
Use existing institutional
ID systems in external
web applications
Integrate file transfer and sharing
capabilities into scientific web
apps, portals, gateways, etc...
Example web apps that leverage Globus
7
PaaS Security Challenges – Globus Auth
• How to provide:
– Login to apps
o Web apps (Jupyter Notebook, Portals), Mobile, Desktop, Command line
– Protect all REST API communications
o App à Globus service (Jupyter Notebook, MRDP)
o App à non-Globus service (MRDP)
o Service à service (MRDP)
• While:
– Not introducing even more identities
o Providing a platform to consolidate those identities
– Providing least privileges security model (consents)
– Being agnostic to programming language and framework
– Being web friendly
– Making it easy for users and developers
8
Client
(Web Portal,
Application)
Globus Transfer
(Resource Server)
Globus Auth
(Authorization
Server)
5. Authenticate using client id
and secret, send authorization
code
Authorization Code Grant
Browser (User)
1. Access
portal
2.
Redirects
user
3. User authenticates and
consents
4. Authorization
code
6. Access token(s)
7. Authenticate with access
token(s) to give the client
the authority invoke the
transfer service
Identity
Provider
Globus Transfer API
• Globus Web App consumes public Transfer API
• Resource named by URL (standard REST approach)
– Query params allow refinement (e.g., subset of fields)
• Globus APIs use JSON for documents and resource
representations
• Requests authorized via OAuth2 access token
– Authorization: Bearer asdflkqhafsdafeawk
docs.globus.org/api/transfer
10
Globus Python SDK
• Python client library for the Globus Auth and Transfer
REST APIs
• globus_sdk.TransferClient class handles
connection management, security, framing,
marshaling
from globus_sdk import TransferClient
tc = TransferClient()
globus.github.io/globus-sdk-python
11
TransferClient low-level calls
• Thin wrapper around REST API
– post(), get(), update(), delete()
get(path, params=None, headers=None, auth=None,
response_class=None)
o path – path for the request, with or without leading slash
o params – dict to be encoded as a query string
o headers – dict of HTTP headers to add to the request
o response_class – class response object, overrides the client’s
default_response_class
o Returns: GlobusHTTPResponse object
12
TransferClient higher-level calls
• One method for each API resource and HTTP verb
• Largely direct mapping to REST API
endpoint_search(filter_fulltext=None,
filter_scope=None,
num_results=25,
**params)
13
Walkthrough API with our Jupyter Hub
• https://jupyter.demo.globus.org
– Sign in with Globus
– Verify the consents
– Start My Server (this will take about a minute)
– Open folder: globus-jupyter-notebooks
– Open folder: GlobusWorldTour
– Run Platform_Introduction_JupyterHub_Auth.ipynb
• If you mess it up and want to “go back to the beginning”
– Back down to the root folder
– Run NotebookPuller.ipynb
• If you want to use the notebook outside of our hub
– https://github.com/globus/globus-jupyter-notebooks
– Authentication is a manual cut and paste of exchanging the authorization
code for an access token
14
Endpoint Search
• Plain text search for endpoint
– Searches owner, display name, keywords, description,
organization, department
– Full word and prefix match
• Limit search to pre-defined scopes
– all, my-endpoints, recently-used, in-use, shared-
by-me, shared-with-me
• Returns: List of endpoint documents
15
Endpoint Management
• Get endpoint (by id)
• Update endpoint
• Create & delete (shared) endpoints
• Manage endpoint servers
16
Endpoint Activation
• Activating endpoint means binding a credential to an
endpoint for login
• Globus Connect Server endpoint that have MyProxy
or MyProxy OAuth identity provider require login via
web
• Auto-activate
– Globus Connect Personal and Shared endpoints use Globus-
provided credential
– Must auto-activate before any API calls to endpoints
17
File operations
• List directory contents (ls)
• Make directory (mkdir)
• Rename
• Note:
– Path encoding & UTF gotchas
– Don’t forget to auto-activate first
18
Task submission
• Asynchronous operations
– Transfer
o Sync level option
– Delete
• Get submission_id, followed by submit
– Once and only once submission
19
Task management
• Get task by id
• Get task_list
• Update task by id (label, deadline)
• Cancel task by id
• Get event list for task
• Get task pause info
20
Bookmarks
• Get list of bookmarks
• Create bookmark
• Get bookmark by id
• Update bookmark
• Delete bookmark by id
• Cannot perform other operations directly on bookmarks
– Requires client-side resolution
21
Shared endpoints and access rules (ACLs)
• Shared Endpoint – create / delete / get info / get list
• Administrator role required to delegate access managers
• Access manager role required to manage
permission/ACLs
• Operations:
– Get list of access rules
– Get access rule by id
– Create access rule
– Update access rule
– Delete access rule
22
Management API
• Allow endpoint administrators to monitor and manage
all tasks with endpoint
– Task API is essentially the same as for users
– Information limited to what they could see locally
• Cancel tasks
• Pause rules
23
Walkthrough API with a Jupyter Notebook
• If you want to use the / a notebook outside of our hub
– https://github.com/globus/globus-jupyter-notebooks
– Authentication is a manual cut and paste of the authorization
code.
– Native App grant
24
Globus Helper Pages
• Globus pages designed for use by your web apps
– Browse Endpoint
– Activate Endpoint
– Select Group
– Manage Identities
– Manage Consents
– Logout
docs.globus.org/api/helper-pages
26
Globus PaaS developer resources
Python SDK
Sample
Application
docs.globus.org/api github.com/globus
Jupyter Notebook
Support resources
• Globus documentation: docs.globus.org
• Helpdesk and issue escalation: support@globus.org
• Mailing lists
– https://www.globus.org/mailing-lists
– developer-discuss@globus.org
• Globus professional services team
– Assist with portal/gateway/app architecture and design
– Develop custom applications that leverage the Globus platform
– Advise on customized deployment and integration scenarios
1 of 27

Recommended

Introduction to the Globus Platform (APS Workshop) by
Introduction to the Globus Platform (APS Workshop)Introduction to the Globus Platform (APS Workshop)
Introduction to the Globus Platform (APS Workshop)Globus
97 views24 slides
Introduction to Globus (GlobusWorld Tour West) by
Introduction to Globus (GlobusWorld Tour West)Introduction to Globus (GlobusWorld Tour West)
Introduction to Globus (GlobusWorld Tour West)Globus
110 views13 slides
Tutorial: Leveraging Globus in your Research Applications by
Tutorial: Leveraging Globus in your Research ApplicationsTutorial: Leveraging Globus in your Research Applications
Tutorial: Leveraging Globus in your Research ApplicationsGlobus
292 views29 slides
Globus Command Line Interface (APS Workshop) by
Globus Command Line Interface (APS Workshop)Globus Command Line Interface (APS Workshop)
Globus Command Line Interface (APS Workshop)Globus
100 views25 slides
Simple Data Automation with Globus (GlobusWorld Tour West) by
Simple Data Automation with Globus (GlobusWorld Tour West)Simple Data Automation with Globus (GlobusWorld Tour West)
Simple Data Automation with Globus (GlobusWorld Tour West)Globus
101 views28 slides
GlobusWorld 2021 Tutorial: Globus for System Administrators by
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobus
141 views31 slides

More Related Content

What's hot

Best Practices for Data Sharing (GlobusWorld Tour - UCSD) by
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)Best Practices for Data Sharing (GlobusWorld Tour - UCSD)
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)Globus
65 views17 slides
Leveraging the Globus Platform in Web Applications (CHPC 2019 - South Africa) by
Leveraging the Globus Platform in Web Applications (CHPC 2019 - South Africa)Leveraging the Globus Platform in Web Applications (CHPC 2019 - South Africa)
Leveraging the Globus Platform in Web Applications (CHPC 2019 - South Africa)Globus
106 views30 slides
Connecting Your System to Globus (APS Workshop) by
Connecting Your System to Globus (APS Workshop)Connecting Your System to Globus (APS Workshop)
Connecting Your System to Globus (APS Workshop)Globus
137 views67 slides
GlobusWorld 2021 Tutorial: Building with the Globus Platform by
GlobusWorld 2021 Tutorial: Building with the Globus PlatformGlobusWorld 2021 Tutorial: Building with the Globus Platform
GlobusWorld 2021 Tutorial: Building with the Globus PlatformGlobus
156 views23 slides
GlobusWorld 2021 Tutorial: Introduction to Globus by
GlobusWorld 2021 Tutorial: Introduction to GlobusGlobusWorld 2021 Tutorial: Introduction to Globus
GlobusWorld 2021 Tutorial: Introduction to GlobusGlobus
158 views13 slides
Globus Portal Framework (APS Workshop) by
Globus Portal Framework (APS Workshop)Globus Portal Framework (APS Workshop)
Globus Portal Framework (APS Workshop)Globus
121 views27 slides

What's hot(20)

Best Practices for Data Sharing (GlobusWorld Tour - UCSD) by Globus
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)Best Practices for Data Sharing (GlobusWorld Tour - UCSD)
Best Practices for Data Sharing (GlobusWorld Tour - UCSD)
Globus 65 views
Leveraging the Globus Platform in Web Applications (CHPC 2019 - South Africa) by Globus
Leveraging the Globus Platform in Web Applications (CHPC 2019 - South Africa)Leveraging the Globus Platform in Web Applications (CHPC 2019 - South Africa)
Leveraging the Globus Platform in Web Applications (CHPC 2019 - South Africa)
Globus 106 views
Connecting Your System to Globus (APS Workshop) by Globus
Connecting Your System to Globus (APS Workshop)Connecting Your System to Globus (APS Workshop)
Connecting Your System to Globus (APS Workshop)
Globus 137 views
GlobusWorld 2021 Tutorial: Building with the Globus Platform by Globus
GlobusWorld 2021 Tutorial: Building with the Globus PlatformGlobusWorld 2021 Tutorial: Building with the Globus Platform
GlobusWorld 2021 Tutorial: Building with the Globus Platform
Globus 156 views
GlobusWorld 2021 Tutorial: Introduction to Globus by Globus
GlobusWorld 2021 Tutorial: Introduction to GlobusGlobusWorld 2021 Tutorial: Introduction to Globus
GlobusWorld 2021 Tutorial: Introduction to Globus
Globus 158 views
Globus Portal Framework (APS Workshop) by Globus
Globus Portal Framework (APS Workshop)Globus Portal Framework (APS Workshop)
Globus Portal Framework (APS Workshop)
Globus 121 views
Automating Research Data Flows with the Globus Command Line Interface (CLI) by Globus
Automating Research Data Flows with the Globus Command Line Interface (CLI)Automating Research Data Flows with the Globus Command Line Interface (CLI)
Automating Research Data Flows with the Globus Command Line Interface (CLI)
Globus 237 views
Globus Platform Overview by Globus
Globus Platform OverviewGlobus Platform Overview
Globus Platform Overview
Globus 260 views
Globus for System Administrators by Globus
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
Globus 268 views
Automating Research Data Flows with Globus (CHPC 2019 - South Africa) by Globus
Automating Research Data Flows with Globus (CHPC 2019 - South Africa)Automating Research Data Flows with Globus (CHPC 2019 - South Africa)
Automating Research Data Flows with Globus (CHPC 2019 - South Africa)
Globus 156 views
Data Orchestration at Scale (GlobusWorld Tour West) by Globus
Data Orchestration at Scale (GlobusWorld Tour West)Data Orchestration at Scale (GlobusWorld Tour West)
Data Orchestration at Scale (GlobusWorld Tour West)
Globus 102 views
Globus: Beyond File Transfer by Globus
Globus: Beyond File TransferGlobus: Beyond File Transfer
Globus: Beyond File Transfer
Globus 304 views
Making Storage Systems Accessible via Globus (GlobusWorld Tour West) by Globus
Making Storage Systems Accessible via Globus (GlobusWorld Tour West)Making Storage Systems Accessible via Globus (GlobusWorld Tour West)
Making Storage Systems Accessible via Globus (GlobusWorld Tour West)
Globus 128 views
Introduction to Globus for New Users (GlobusWorld Tour - UCSD) by Globus
Introduction to Globus for New Users (GlobusWorld Tour - UCSD)Introduction to Globus for New Users (GlobusWorld Tour - UCSD)
Introduction to Globus for New Users (GlobusWorld Tour - UCSD)
Globus 64 views
GlobusWorld 2020 Keynote by Globus
GlobusWorld 2020 KeynoteGlobusWorld 2020 Keynote
GlobusWorld 2020 Keynote
Globus 434 views
Introduction to the Globus PaaS (GlobusWorld Tour - STFC) by Globus
Introduction to the Globus PaaS (GlobusWorld Tour - STFC)Introduction to the Globus PaaS (GlobusWorld Tour - STFC)
Introduction to the Globus PaaS (GlobusWorld Tour - STFC)
Globus 77 views
Gateways 2020 Tutorial - Introduction to Globus by Globus
Gateways 2020 Tutorial - Introduction to GlobusGateways 2020 Tutorial - Introduction to Globus
Gateways 2020 Tutorial - Introduction to Globus
Globus 151 views
Tutorial: Managing Protected Data with Globus Connect Server v5 by Globus
Tutorial: Managing Protected Data with Globus Connect Server v5Tutorial: Managing Protected Data with Globus Connect Server v5
Tutorial: Managing Protected Data with Globus Connect Server v5
Globus 371 views
Gateways 2020 Tutorial - Large Scale Data Transfer with Globus by Globus
Gateways 2020 Tutorial - Large Scale Data Transfer with GlobusGateways 2020 Tutorial - Large Scale Data Transfer with Globus
Gateways 2020 Tutorial - Large Scale Data Transfer with Globus
Globus 137 views
Gateways 2020 Tutorial - Automated Data Ingest and Search with Globus by Globus
Gateways 2020 Tutorial - Automated Data Ingest and Search with GlobusGateways 2020 Tutorial - Automated Data Ingest and Search with Globus
Gateways 2020 Tutorial - Automated Data Ingest and Search with Globus
Globus 203 views

Similar to Introduction to the Globus Platform (GlobusWorld Tour - UMich)

Leveraging the Globus Platform (GlobusWorld Tour - Columbia University) by
Leveraging the Globus Platform (GlobusWorld Tour - Columbia University)Leveraging the Globus Platform (GlobusWorld Tour - Columbia University)
Leveraging the Globus Platform (GlobusWorld Tour - Columbia University)Globus
84 views42 slides
Leveraging the Globus Platform (GlobusWorld Tour - UCSD) by
Leveraging the Globus Platform (GlobusWorld Tour - UCSD)Leveraging the Globus Platform (GlobusWorld Tour - UCSD)
Leveraging the Globus Platform (GlobusWorld Tour - UCSD)Globus
62 views26 slides
Data Publication and Discovery with Globus by
Data Publication and Discovery with GlobusData Publication and Discovery with Globus
Data Publication and Discovery with GlobusGlobus
266 views76 slides
Automating Research Data Flows and Introduction to the Globus Platform by
Automating Research Data Flows and Introduction to the Globus PlatformAutomating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus PlatformGlobus
50 views41 slides
Jupyter + Globus: The Foundation for Interactive Data Science by
Jupyter + Globus: The Foundation for Interactive Data ScienceJupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data ScienceGlobus
423 views99 slides
Automating Research Data Flows and an Introduction to the Globus Platform by
Automating Research Data Flows and an Introduction to the Globus PlatformAutomating Research Data Flows and an Introduction to the Globus Platform
Automating Research Data Flows and an Introduction to the Globus PlatformGlobus
132 views42 slides

Similar to Introduction to the Globus Platform (GlobusWorld Tour - UMich)(20)

Leveraging the Globus Platform (GlobusWorld Tour - Columbia University) by Globus
Leveraging the Globus Platform (GlobusWorld Tour - Columbia University)Leveraging the Globus Platform (GlobusWorld Tour - Columbia University)
Leveraging the Globus Platform (GlobusWorld Tour - Columbia University)
Globus 84 views
Leveraging the Globus Platform (GlobusWorld Tour - UCSD) by Globus
Leveraging the Globus Platform (GlobusWorld Tour - UCSD)Leveraging the Globus Platform (GlobusWorld Tour - UCSD)
Leveraging the Globus Platform (GlobusWorld Tour - UCSD)
Globus 62 views
Data Publication and Discovery with Globus by Globus
Data Publication and Discovery with GlobusData Publication and Discovery with Globus
Data Publication and Discovery with Globus
Globus 266 views
Automating Research Data Flows and Introduction to the Globus Platform by Globus
Automating Research Data Flows and Introduction to the Globus PlatformAutomating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus Platform
Globus 50 views
Jupyter + Globus: The Foundation for Interactive Data Science by Globus
Jupyter + Globus: The Foundation for Interactive Data ScienceJupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data Science
Globus 423 views
Automating Research Data Flows and an Introduction to the Globus Platform by Globus
Automating Research Data Flows and an Introduction to the Globus PlatformAutomating Research Data Flows and an Introduction to the Globus Platform
Automating Research Data Flows and an Introduction to the Globus Platform
Globus 132 views
Working with Globus Platform Services and Portals by Globus
Working with Globus Platform Services and PortalsWorking with Globus Platform Services and Portals
Working with Globus Platform Services and Portals
Globus 28 views
Building Data Portals and Science Gateways with Globus by Globus
Building Data Portals and Science Gateways with GlobusBuilding Data Portals and Science Gateways with Globus
Building Data Portals and Science Gateways with Globus
Globus 133 views
Introduction to Globus: Research Data Management Software at the ALCF by Globus
Introduction to Globus: Research Data Management Software at the ALCFIntroduction to Globus: Research Data Management Software at the ALCF
Introduction to Globus: Research Data Management Software at the ALCF
Globus 274 views
Working with Globus Platform Services by Globus
Working with Globus Platform ServicesWorking with Globus Platform Services
Working with Globus Platform Services
Globus 41 views
"What's New With Globus" Webinar: Spring 2018 by Globus
"What's New With Globus" Webinar: Spring 2018"What's New With Globus" Webinar: Spring 2018
"What's New With Globus" Webinar: Spring 2018
Globus 270 views
Globus: Research Data Management as Service and Platform - pearc17 by Mary Bass
Globus: Research Data Management as Service and Platform - pearc17Globus: Research Data Management as Service and Platform - pearc17
Globus: Research Data Management as Service and Platform - pearc17
Mary Bass129 views
Automating Research Data Workflows (GlobusWorld Tour - Columbia University) by Globus
Automating Research Data Workflows (GlobusWorld Tour - Columbia University)Automating Research Data Workflows (GlobusWorld Tour - Columbia University)
Automating Research Data Workflows (GlobusWorld Tour - Columbia University)
Globus 120 views
Getting Started with Globus for Developers by Globus
Getting Started with Globus for DevelopersGetting Started with Globus for Developers
Getting Started with Globus for Developers
Globus 111 views
Introduction to Globus for New Users by Globus
Introduction to Globus for New UsersIntroduction to Globus for New Users
Introduction to Globus for New Users
Globus 58 views
Automating Data Flows with the Globus CLI (GlobusWorld Tour - UMich) by Globus
Automating Data Flows with the Globus CLI (GlobusWorld Tour - UMich)Automating Data Flows with the Globus CLI (GlobusWorld Tour - UMich)
Automating Data Flows with the Globus CLI (GlobusWorld Tour - UMich)
Globus 231 views
Tutorial: Automating Research Data Workflows by Globus
Tutorial: Automating Research Data WorkflowsTutorial: Automating Research Data Workflows
Tutorial: Automating Research Data Workflows
Globus 135 views
Facilitating Collaboration with Globus (GlobusWorld Tour - STFC) by Globus
Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)
Facilitating Collaboration with Globus (GlobusWorld Tour - STFC)
Globus 125 views
Globus for System Administrators (CHPC 2019 - South Africa) by Globus
Globus for System Administrators (CHPC 2019 - South Africa)Globus for System Administrators (CHPC 2019 - South Africa)
Globus for System Administrators (CHPC 2019 - South Africa)
Globus 180 views
Tutorial: Introduction to Globus for System Administrators by Globus
Tutorial: Introduction to Globus for System AdministratorsTutorial: Introduction to Globus for System Administrators
Tutorial: Introduction to Globus for System Administrators
Globus 465 views

More from Globus

Introduction to Globus for System Administrators by
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System AdministratorsGlobus
11 views55 slides
Introduction to Data Transfer and Sharing for Researchers by
Introduction to Data Transfer and Sharing for ResearchersIntroduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for ResearchersGlobus
4 views33 slides
Introduction to the Globus Platform for Developers by
Introduction to the Globus Platform for DevelopersIntroduction to the Globus Platform for Developers
Introduction to the Globus Platform for DevelopersGlobus
4 views28 slides
Introduction to the Command Line Interface (CLI) by
Introduction to the Command Line Interface (CLI)Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)Globus
12 views12 slides
Automating Research Data with Globus Flows and Compute by
Automating Research Data with Globus Flows and ComputeAutomating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and ComputeGlobus
6 views60 slides
Advanced Globus System Administration by
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System AdministrationGlobus
26 views29 slides

More from Globus (20)

Introduction to Globus for System Administrators by Globus
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
Globus 11 views
Introduction to Data Transfer and Sharing for Researchers by Globus
Introduction to Data Transfer and Sharing for ResearchersIntroduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for Researchers
Globus 4 views
Introduction to the Globus Platform for Developers by Globus
Introduction to the Globus Platform for DevelopersIntroduction to the Globus Platform for Developers
Introduction to the Globus Platform for Developers
Globus 4 views
Introduction to the Command Line Interface (CLI) by Globus
Introduction to the Command Line Interface (CLI)Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)
Globus 12 views
Automating Research Data with Globus Flows and Compute by Globus
Automating Research Data with Globus Flows and ComputeAutomating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and Compute
Globus 6 views
Advanced Globus System Administration by Globus
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
Globus 26 views
Introduction to Globus for System Administrators by Globus
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
Globus 94 views
Introduction to Globus for New Users by Globus
Introduction to Globus for New UsersIntroduction to Globus for New Users
Introduction to Globus for New Users
Globus 55 views
Globus Automation by Globus
Globus AutomationGlobus Automation
Globus Automation
Globus 20 views
Advanced Globus System Administration by Globus
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
Globus 21 views
Introduction to Globus by Globus
Introduction to GlobusIntroduction to Globus
Introduction to Globus
Globus 38 views
Introduction to Globus for System Administrators by Globus
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
Globus 27 views
Advanced Globus System Administration by Globus
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
Globus 29 views
Introduction to Globus for System Administrators by Globus
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
Globus 145 views
Using Globus to Streamline Research at Scale by Globus
Using Globus to Streamline Research at ScaleUsing Globus to Streamline Research at Scale
Using Globus to Streamline Research at Scale
Globus 30 views
Introduction to Globus for Researchers by Globus
Introduction to Globus for ResearchersIntroduction to Globus for Researchers
Introduction to Globus for Researchers
Globus 89 views
Globus Endpoint Migration and Advanced Administration Topics by Globus
Globus Endpoint Migration and Advanced Administration TopicsGlobus Endpoint Migration and Advanced Administration Topics
Globus Endpoint Migration and Advanced Administration Topics
Globus 55 views
Globus for System Administrators by Globus
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
Globus 68 views
Automating Research Data Management with Globus by Globus
Automating Research Data Management with GlobusAutomating Research Data Management with Globus
Automating Research Data Management with Globus
Globus 250 views
Moemoea nui Aotearoa: Challenges and Strategies in Data Lifecycle Management ... by Globus
Moemoea nui Aotearoa: Challenges and Strategies in Data Lifecycle Management ...Moemoea nui Aotearoa: Challenges and Strategies in Data Lifecycle Management ...
Moemoea nui Aotearoa: Challenges and Strategies in Data Lifecycle Management ...
Globus 150 views

Recently uploaded

[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation by
[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation
[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented GenerationDataScienceConferenc1
7 views29 slides
Cross-network in Google Analytics 4.pdf by
Cross-network in Google Analytics 4.pdfCross-network in Google Analytics 4.pdf
Cross-network in Google Analytics 4.pdfGA4 Tutorials
6 views7 slides
Building Real-Time Travel Alerts by
Building Real-Time Travel AlertsBuilding Real-Time Travel Alerts
Building Real-Time Travel AlertsTimothy Spann
111 views48 slides
Survey on Factuality in LLM's.pptx by
Survey on Factuality in LLM's.pptxSurvey on Factuality in LLM's.pptx
Survey on Factuality in LLM's.pptxNeethaSherra1
5 views9 slides
Short Story Assignment by Kelly Nguyen by
Short Story Assignment by Kelly NguyenShort Story Assignment by Kelly Nguyen
Short Story Assignment by Kelly Nguyenkellynguyen01
19 views17 slides
CRIJ4385_Death Penalty_F23.pptx by
CRIJ4385_Death Penalty_F23.pptxCRIJ4385_Death Penalty_F23.pptx
CRIJ4385_Death Penalty_F23.pptxyvettemm100
6 views24 slides

Recently uploaded(20)

[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation by DataScienceConferenc1
[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation
[DSC Europe 23] Spela Poklukar & Tea Brasanac - Retrieval Augmented Generation
Cross-network in Google Analytics 4.pdf by GA4 Tutorials
Cross-network in Google Analytics 4.pdfCross-network in Google Analytics 4.pdf
Cross-network in Google Analytics 4.pdf
GA4 Tutorials6 views
Building Real-Time Travel Alerts by Timothy Spann
Building Real-Time Travel AlertsBuilding Real-Time Travel Alerts
Building Real-Time Travel Alerts
Timothy Spann111 views
Survey on Factuality in LLM's.pptx by NeethaSherra1
Survey on Factuality in LLM's.pptxSurvey on Factuality in LLM's.pptx
Survey on Factuality in LLM's.pptx
NeethaSherra15 views
Short Story Assignment by Kelly Nguyen by kellynguyen01
Short Story Assignment by Kelly NguyenShort Story Assignment by Kelly Nguyen
Short Story Assignment by Kelly Nguyen
kellynguyen0119 views
CRIJ4385_Death Penalty_F23.pptx by yvettemm100
CRIJ4385_Death Penalty_F23.pptxCRIJ4385_Death Penalty_F23.pptx
CRIJ4385_Death Penalty_F23.pptx
yvettemm1006 views
UNEP FI CRS Climate Risk Results.pptx by pekka28
UNEP FI CRS Climate Risk Results.pptxUNEP FI CRS Climate Risk Results.pptx
UNEP FI CRS Climate Risk Results.pptx
pekka2811 views
[DSC Europe 23] Zsolt Feleki - Machine Translation should we trust it.pptx by DataScienceConferenc1
[DSC Europe 23] Zsolt Feleki - Machine Translation should we trust it.pptx[DSC Europe 23] Zsolt Feleki - Machine Translation should we trust it.pptx
[DSC Europe 23] Zsolt Feleki - Machine Translation should we trust it.pptx
3196 The Case of The East River by ErickANDRADE90
3196 The Case of The East River3196 The Case of The East River
3196 The Case of The East River
ErickANDRADE9011 views
Understanding Hallucinations in LLMs - 2023 09 29.pptx by Greg Makowski
Understanding Hallucinations in LLMs - 2023 09 29.pptxUnderstanding Hallucinations in LLMs - 2023 09 29.pptx
Understanding Hallucinations in LLMs - 2023 09 29.pptx
Greg Makowski17 views
Chapter 3b- Process Communication (1) (1)(1) (1).pptx by ayeshabaig2004
Chapter 3b- Process Communication (1) (1)(1) (1).pptxChapter 3b- Process Communication (1) (1)(1) (1).pptx
Chapter 3b- Process Communication (1) (1)(1) (1).pptx
ayeshabaig20045 views
Organic Shopping in Google Analytics 4.pdf by GA4 Tutorials
Organic Shopping in Google Analytics 4.pdfOrganic Shopping in Google Analytics 4.pdf
Organic Shopping in Google Analytics 4.pdf
GA4 Tutorials11 views
Data structure and algorithm. by Abdul salam
Data structure and algorithm. Data structure and algorithm.
Data structure and algorithm.
Abdul salam 19 views
Advanced_Recommendation_Systems_Presentation.pptx by neeharikasingh29
Advanced_Recommendation_Systems_Presentation.pptxAdvanced_Recommendation_Systems_Presentation.pptx
Advanced_Recommendation_Systems_Presentation.pptx
Supercharging your Data with Azure AI Search and Azure OpenAI by Peter Gallagher
Supercharging your Data with Azure AI Search and Azure OpenAISupercharging your Data with Azure AI Search and Azure OpenAI
Supercharging your Data with Azure AI Search and Azure OpenAI
Peter Gallagher37 views
Vikas 500 BIG DATA TECHNOLOGIES LAB.pdf by vikas12611618
Vikas 500 BIG DATA TECHNOLOGIES LAB.pdfVikas 500 BIG DATA TECHNOLOGIES LAB.pdf
Vikas 500 BIG DATA TECHNOLOGIES LAB.pdf
vikas126116188 views

Introduction to the Globus Platform (GlobusWorld Tour - UMich)

  • 1. Introduction to the Globus Platform Greg Nawrocki nawrocki@uchicago.edu greg@globus.org University of Michigan July 23, 2019
  • 3. 3 Globus delivers… with applications and as a platform… Fast and reliable data transfer, sharing, and file management… …directly from your own storage systems… ...via software-as-a-service using existing identities.
  • 4. How can I integrate Globus into my research workflows? 4
  • 5. Globus serves as… A platform for building science gateways, portals and other web applications in support of research and education. 5
  • 6. Globus Auth API (Group Management) … GlobusTransferAPI GlobusConnect File Sharing File Transfer & Replication Globus Platform-as-a-Service Use existing institutional ID systems in external web applications Integrate file transfer and sharing capabilities into scientific web apps, portals, gateways, etc...
  • 7. Example web apps that leverage Globus 7
  • 8. PaaS Security Challenges – Globus Auth • How to provide: – Login to apps o Web apps (Jupyter Notebook, Portals), Mobile, Desktop, Command line – Protect all REST API communications o App à Globus service (Jupyter Notebook, MRDP) o App à non-Globus service (MRDP) o Service à service (MRDP) • While: – Not introducing even more identities o Providing a platform to consolidate those identities – Providing least privileges security model (consents) – Being agnostic to programming language and framework – Being web friendly – Making it easy for users and developers 8
  • 9. Client (Web Portal, Application) Globus Transfer (Resource Server) Globus Auth (Authorization Server) 5. Authenticate using client id and secret, send authorization code Authorization Code Grant Browser (User) 1. Access portal 2. Redirects user 3. User authenticates and consents 4. Authorization code 6. Access token(s) 7. Authenticate with access token(s) to give the client the authority invoke the transfer service Identity Provider
  • 10. Globus Transfer API • Globus Web App consumes public Transfer API • Resource named by URL (standard REST approach) – Query params allow refinement (e.g., subset of fields) • Globus APIs use JSON for documents and resource representations • Requests authorized via OAuth2 access token – Authorization: Bearer asdflkqhafsdafeawk docs.globus.org/api/transfer 10
  • 11. Globus Python SDK • Python client library for the Globus Auth and Transfer REST APIs • globus_sdk.TransferClient class handles connection management, security, framing, marshaling from globus_sdk import TransferClient tc = TransferClient() globus.github.io/globus-sdk-python 11
  • 12. TransferClient low-level calls • Thin wrapper around REST API – post(), get(), update(), delete() get(path, params=None, headers=None, auth=None, response_class=None) o path – path for the request, with or without leading slash o params – dict to be encoded as a query string o headers – dict of HTTP headers to add to the request o response_class – class response object, overrides the client’s default_response_class o Returns: GlobusHTTPResponse object 12
  • 13. TransferClient higher-level calls • One method for each API resource and HTTP verb • Largely direct mapping to REST API endpoint_search(filter_fulltext=None, filter_scope=None, num_results=25, **params) 13
  • 14. Walkthrough API with our Jupyter Hub • https://jupyter.demo.globus.org – Sign in with Globus – Verify the consents – Start My Server (this will take about a minute) – Open folder: globus-jupyter-notebooks – Open folder: GlobusWorldTour – Run Platform_Introduction_JupyterHub_Auth.ipynb • If you mess it up and want to “go back to the beginning” – Back down to the root folder – Run NotebookPuller.ipynb • If you want to use the notebook outside of our hub – https://github.com/globus/globus-jupyter-notebooks – Authentication is a manual cut and paste of exchanging the authorization code for an access token 14
  • 15. Endpoint Search • Plain text search for endpoint – Searches owner, display name, keywords, description, organization, department – Full word and prefix match • Limit search to pre-defined scopes – all, my-endpoints, recently-used, in-use, shared- by-me, shared-with-me • Returns: List of endpoint documents 15
  • 16. Endpoint Management • Get endpoint (by id) • Update endpoint • Create & delete (shared) endpoints • Manage endpoint servers 16
  • 17. Endpoint Activation • Activating endpoint means binding a credential to an endpoint for login • Globus Connect Server endpoint that have MyProxy or MyProxy OAuth identity provider require login via web • Auto-activate – Globus Connect Personal and Shared endpoints use Globus- provided credential – Must auto-activate before any API calls to endpoints 17
  • 18. File operations • List directory contents (ls) • Make directory (mkdir) • Rename • Note: – Path encoding & UTF gotchas – Don’t forget to auto-activate first 18
  • 19. Task submission • Asynchronous operations – Transfer o Sync level option – Delete • Get submission_id, followed by submit – Once and only once submission 19
  • 20. Task management • Get task by id • Get task_list • Update task by id (label, deadline) • Cancel task by id • Get event list for task • Get task pause info 20
  • 21. Bookmarks • Get list of bookmarks • Create bookmark • Get bookmark by id • Update bookmark • Delete bookmark by id • Cannot perform other operations directly on bookmarks – Requires client-side resolution 21
  • 22. Shared endpoints and access rules (ACLs) • Shared Endpoint – create / delete / get info / get list • Administrator role required to delegate access managers • Access manager role required to manage permission/ACLs • Operations: – Get list of access rules – Get access rule by id – Create access rule – Update access rule – Delete access rule 22
  • 23. Management API • Allow endpoint administrators to monitor and manage all tasks with endpoint – Task API is essentially the same as for users – Information limited to what they could see locally • Cancel tasks • Pause rules 23
  • 24. Walkthrough API with a Jupyter Notebook • If you want to use the / a notebook outside of our hub – https://github.com/globus/globus-jupyter-notebooks – Authentication is a manual cut and paste of the authorization code. – Native App grant 24
  • 25. Globus Helper Pages • Globus pages designed for use by your web apps – Browse Endpoint – Activate Endpoint – Select Group – Manage Identities – Manage Consents – Logout docs.globus.org/api/helper-pages 26
  • 26. Globus PaaS developer resources Python SDK Sample Application docs.globus.org/api github.com/globus Jupyter Notebook
  • 27. Support resources • Globus documentation: docs.globus.org • Helpdesk and issue escalation: support@globus.org • Mailing lists – https://www.globus.org/mailing-lists – developer-discuss@globus.org • Globus professional services team – Assist with portal/gateway/app architecture and design – Develop custom applications that leverage the Globus platform – Advise on customized deployment and integration scenarios