1. Globus - Beyond
File Transfer
A Globus Introduction and
more!
Greg Nawrocki
greg@globus.org
nawrocki@uchicago.edu

2. Research data management today
How do we...
...move?
...share?
...discover?
...reproduce?
Index?

3. 3
Globus delivers…
Fast and reliable big data transfer,
sharing, and platform services…
…directly from your own storage
systems…
...via software-as-a-service using
existing identities with the overarching
goal of...

4. 4
Research Computing HPC
Desktop Workstations
Mass Storage Instruments
Personal Resources
Public Cloud
National Resources
Unifying access to data across tiers

5. Globus Connectors
ActiveScale
Object
Storage
Planned

6. Public / private cloud stores
External
campus
storage
EC2
Project
repositories,
replication stores
Public repositories
Share with collaborators/community

7. Analysis
store
Next-Gen Sequencer
MRI
Advanced Light Source
Personal system
Remote visualization
Light Sheet Microscope
High-durability,
low-cost store
Manage data from instruments
Cryo-EM

8. Use(r)-appropriate interfaces
8
GET /endpoint/go%23ep1
PUT /endpoint/vas#my_endpt
200 OK
X-Transfer-API-Version: 0.10
Content-Type: application/json
…
Globus service
Web
CLI
Rest
API

9. Globus SaaS / PaaS: Research data lifecycle
Researcher initiates
transfer request; or
requested automatically
by script, science
gateway
1
Instrument
Compute Facility
Globus transfers files
reliably, securely
2
Globus controls
access to shared
files on existing
storage; no need
to move files to
cloud storage!
4
Researcher
selects files to
share, selects
user or group,
and sets access
permissions
3
Collaborator logs in to
Globus and accesses
shared files; no local
account required;
download via Globus
5
Automating research
workflows and
ensuring those that
need access to the
data have it.
8
Personal Computer
Transfer
Share
• Use a Web browser or
platform services
• Access any storage
• Use an existing identity
Build
The Globus
Command Line
Interface, API sets,
and Python SDK
provide a platform…
6
… for building
science gateways,
portals and
publication services.
7

10. Conceptual architecture: Hybrid SaaS
DATA
Channel
CONTROL
Channel
Source
Endpoint
Destination
Endpoint
Subscriber owned
and administered
storage system
Globus
“client” software
No data relay or
staging via Globus
Subscriber
Control
Domain
Globus
Control
Domain
Single, globally accessible
multi-tenant service
User
accessing
the Globus
Web App via
a browser

11. Conceptual architecture: Sharing
Managed
Endpoint
Subscriber
Control
Domain
Globus
Control
Domain Globus managed
”overlay” permissions
Shared
Endpoint
DATA
Channel
CONTROL
Channel Subscriber managed
filesystem permissions
External User
Control
Domain

12. …makes your
storage system a
Globus endpoint

13. Endpoints (Collections)
• Storage abstraction
– All transfers happen between two endpoints
– Globus Connect instantiates endpoints
• Collection ~= Endpoint
• Test / Demo Endpoints
– Globus Tutorial Endpoint 1
– Globus Tutorial Endpoint 2
– ESnet Test Endpoints
o Contain file samples of various sizes
• Globus Connect Personal
– Now your laptop is an endpoint
– https://www.globus.org/globus-connect-personal
14

14. Globus Connect Personal
• Installers do not require admin access
• Zero configuration; auto updating
• Handles NATs
• Installs in seconds – easy to delete - I’ll prove it!

15. Almost demo time…
• How do I get a Globus account?
– A Globus Account is
o A Primary Identity
o Possible Linked Identities
– Your existing institutional identity may already work
– Linking / Managing Identities
– Consents
• But I don’t have any Endpoints (Collections)!
– Globus Connect Personal
– Globus Tutorial Endpoint 1
– Globus Tutorial Endpoint 2
– ESnet Test Endpoints
o Contain file samples of various sizes
16

16. Demo time!
Identities and
Accounts Transfer
Sharing
Transfer Details
Bookmarks
The Console
The Hamburger
Menu
The Activity Monitor
Groups
Roles
Responsive
Interface

17. Globus Command Line Interface
Open source, uses
Python SDK
docs.globus.org/cli
github.com/globus/
globus-cli

18. Globus Auth API
(Group Management)
…
GlobusTransferAPI
GlobusConnect
Data Discovery
File Sharing
File Transfer & Replication
Globus Platform-as-a-Service
Use existing institutional
ID systems in external
web applications
Integrate file transfer and sharing
capabilities into scientific web
apps, portals, gateways, etc...

19. A bit of Globus history
U . S . D E P A R T M E N T O F
ENERGY

20. Globus sustainability model
• Standard Subscription
– Shared endpoints
– Management console
– Usage reporting
– Priority support
– Application integration
– HTTPS support
– Branded Web Site
• Premium Storage Connectors
• Alternate Identity Provider (InCommon is standard)
21

21. The path to sustainability

22. Globus by the numbers...
6764
active shared
endpoints
115
subscribers
887+ PB
moved
23,818
active personal
endpoints
94 billion
files processed
1866
active server
endpoints
80
countries where
Globus is used
2.9 PB
largest single
transfer to date
99.9%
availability
745
identity providers
1960
most shared
endpoints
at a single
institution 112,105
total users

23. Manage Protected Data
24
Higher assurance levels for HIPAA and other regulated data
• Support for protected data
such as health related
information
• Share data with collaborators
while meeting compliance
requirements
• Includes BAA option

24. Globus for high assurance data management
• Restricted data handling
– PII (Personally identifiable information)
– Controlled Unclassified Information
– PHI (Protected Health Information)
• University of Chicago security controls
– NIST 800-53 Low
– Superset of 800-171 Low
• Business Associate Agreements (BAA) will be between
University of Chicago and our subscribers
– University of Chicago has a BAA with Amazon

25. High Assurance features
• Additional authentication assurance
– Per storage gateway policy on frequency of authentication with
specific identity for access to data (timeout)
– Ensure that user authenticates with the specific identity that
gives them access within session (decoupling linked identities)
• Session/device isolation
– Authentication context is per application, per session (~browser
session)
• Enforces encryption of all user data in transit
• Audit logging

26. Support resources
• Globus documentation: docs.globus.org
• Helpdesk and issue escalation: support@globus.org
• Mailing Lists
– https://www.globus.org/mailing-lists
• Customer engagement team
• Globus professional services team
– Assist with portal/gateway/app architecture and design
– Develop custom applications that leverage the Globus platform
– Advise on customized deployment and integration scenarios

27. Globus on your Campus
• Webinars
• Programs
– Helping you evangelize Globus within your
institution.
• Professional Services
• Globus World Tour
– Taking the show on the road.
28