SlideShare a Scribd company logo
Globus for System Administrators
Vas Vasiliadis
vas@uchicago.edu
May 2, 2019
Globus Connect Server
2
• Makes your storage accessible via Globus
• Multi-user server, installed and managed by sysadmin
docs.globus.org/globus-connect-server-installation-guide/
Local system users
Local Storage System
(HPC cluster, NAS, …)
Globus Connect Server
MyProxy
CA
GridFTP
Server
OAuth
Server
DTN
• Default access for
all local accounts
• Native packaging
Linux: DEB, RPM
Globus Connect Server
3
Local system users
Local Storage System
(HPC cluster, NAS, …)
Globus Connect Server
MyProxy
CA
GridFTP
Server
OAuth
Server
DTN
Non-POSIX Connectors
POSIX-compliant Connector
server
IBM Spectrum Scale
Current Planned
Storage Connectors - globus.org/connectors
Globus Connect Server v5 Milestones
v5.0: Google
Drive
v5.1: POSIX guest
collections, HTTPS
v5.x+: v4 feature
parity+v5.x: …
• Multi DTN support
Additional connectors
• Custom IdPs
• …
Other
features
v5.2: High assurance
v5.3: Version Unification
By default, assume you
should use GCS v4
Which version of Globus Connect Server do I use?
Which version of Globus Connect Server do I use?
Are you a Globus
subscriber?
Do you have a High Assurance or BAA
subscription (and plan to set up a high
assurance endpoint)?
Use
GCS v4
Use
GCS v5.3
No
Yes
No
Yes
Would you like to support multiple storage
system types on one endpoint?
Do you want to enable HTTPS access to
your storage?
Do you plan to use the S3, Box, Google
Drive or Ceph connectors?
No
No
No
Yes
Yes
Yes
Creating a Globus endpoint on your server
• In this example, Server = Amazon EC2 instance
• Installation and configuration of Globus Connect
Server requires a Globus ID
• Go to globusid.org
• Click “create a Globus ID”
– Optional: associate it with your Globus account
8
What we are going to do:
Install Globus Connect Server
• Access server as user “campusadmin”
• Update repo
• Install package
• Setup Globus Connect Server
Server
(AWS EC2)
ssh
ec2-22-23-24-25
Log into Globus
Transfer a file
1
2
3
Access the newly created
endpoint (as user ‘researcher’)
4
9
Access your server
• Get the IP address for your EC2 server (bit.ly/ec2ip)
• Log in as user ‘campusadmin’
ssh campusadmin@<EC2_instance_IP_address>
• Please sudo su before continuing
– User ‘campusadmin’ has passwordless sudo privileges
10
Install Globus Connect Server
$ sudo su
$ curl –LOs
http://downloads.globus.org/toolkit/globus-connect-
server/globus-connect-server-repo_latest_all.deb
$ dpkg –i globus-connect-server-repo_latest_all.deb
$ apt-get update
$ apt-get -y install globus-connect-server
$ globus-connect-server-setup
You have a working Globus endpoint!
Use your Globus ID username and
password when prompted
Access the Globus endpoint
• Go to Manage Data à Transfer Files
• Access the endpoint you just created
– Search for your EC2 host name in the Endpoint field
– Log in as “researcher”; you will see the user’s home directory
• Transfer files between a test endpoint (e.g. ESnet
read-only) and your EC2 endpoint
12
Globus accounts and endpoint access
• Globus account: Primary identity (+ Linked Identities)
• Endpoint initially accessible by creator
• Endpoint not visible?
– Primary identity is your institutional ID?
– Link your Globus ID!
Configuring Globus
Connect Server
14
Endpoint configuration
• On the Globus service: app.globus.org/endpoints
• On your DTN: /etc/globus-connect-server.conf
– Standard .ini format: [Section] Option = Value
– To enable changes run globus-connect-server-setup
– “Rinse and repeat”
15
Common configuration options
• Endpoints page
– Display Name
– Visibility
– Encryption
• DTN configuration file
– RestrictPaths
– Sharing
– IdentityMethod (CILogon, Oauth)
– SharingRestrictPaths
Exercise: Make your endpoint visible
• Edit endpoint attributes
– Change the name to something useful, e.g. <your_name> EC2
Endpoint
– For the “Visible To” attribute select “Public - Visible to all users”
• Find your neighbor’s endpoint
– Thanks to our superb security …you can access it too J
17
Path Restriction
• Default configuration:
– All paths allowed, access control handled by the OS
• Use RestrictPaths to customize
– Specifies a comma separated list of full paths that clients may access
– Each path may be prefixed by R (read) and/or W (write), or N (none) to explicitly
deny access to a path
– '~’ for authenticated user’s home directory, and * may be used for simple
wildcard matching.
• e.g. Full access to home directory, read access to /data:
– RestrictPaths = RW~,R/data
• e.g. Full access to home directory, deny hidden files:
– RestrictPaths = RW~,N~/.*
18
Exercise: Restrict access
• Set RestrictPaths=RW~,N~/archive
• Run globus-connect-server-setup
• Access your endpoint as ‘researcher’
• What’s changed?
19
• In config file, set Sharing=True
• Run globus-connect-server-setup
• Flag endpoint as “managed” (in web app or via CLI)
* Note: Creation of shared endpoints requires a
Globus subscription for the managed endpoint
Enabling sharing on an endpoint
20
Limit sharing to specific accounts
• SharingUsersAllow =
• SharingGroupsAllow =
• SharingUsersDeny =
• SharingGroupsDeny =
21
Sharing Path Restriction
• Restrict paths where users can create shared endpoints
• Use SharingRestrictPaths to customize
– Same syntax as RestrictPaths
• e.g. Full access to home directory, deny hidden files:
– SharingRestrictPaths = RW~,N~/.*
• e.g. Full access to public folder under home directory:
– SharingRestrictPaths = RW~/public
• e.g. Full access to /proj, read access to /scratch:
– SharingRestrictPaths = RW/proj,R/scratch
22
Endpoint Access
Control/Activation
23
Ports needed for Globus
• Inbound: 2811 (control channel)
• Inbound: 7512 (MyProxy), 443 (OAuth)
• Inbound: 50000-51000 (data channel)
• If restricting outbound connections, allow
connections on:
– 80, 2223 (used during install/config)
– 50000-51000 (GridFTP data channel)
24
EndpointactivationusingMyProxy
Default
configuration
(avoid if at all
possible)
EndpointactivationusingMyProxyOAuth
Best practice
configuration
Single Sign-On with InCommon/CILogon
• Your Shibboleth server must release R&S attributes to
CILogon—especially the ePPN attribute
• Local account must match institutional ID (InCommon ID)
– Test by creating a local user with same name
• In /etc/globus-connect-server.conf set:
AuthorizationMethod = CILogon
CILogonIdentityProvider =
<institution_listed_in_CILogon_IdP_list>
27
Managed endpoints and
subscriptions
28
Subscription configuration
• Subscription manager
– Create/upgrade managed endpoints
– Requires Globus ID linked to Globus account
• Management console permissions
– Independent of subscription manager
– Map managed endpoint to Globus ID
• Globus Plus group
– Subscription Manager is admin
– Can grant admin rights to other members
29
Creating managed endpoints
• Required for sharing, management console, reporting, …
• Convert existing endpoint to managed via CLI (or web):
globus endpoint update --managed <endpt_uuid>
• Must be run by subscription manager
• Important: Re-run endpoint update after deleting/re-
creating endpoint
30
Monitoring and managing
Globus endpoint activity
31
Management console
• Monitor all transfers
• Pause/resume specific transfers
• Add pause conditions with various options
• Resume specific tasks overriding pause conditions
• Cancel tasks
• View sharing ACLs
32
Endpoint Roles
• Administrator: define endpoint and roles
• Access Manager: manage permissions
• Activity Manager: perform control tasks
• Activity Monitor: view activity
33
Demonstration:
Management console
Endpoint Roles
Usage Reporting
34
…on performance
35
Balance: performance - reliability
• Network use parameters: concurrency, parallelism
• Maximum, Preferred values for each
• Transfer considers source and destination endpoint settings
min(
max(preferred src, preferred dest),
max src,
max dest
)
• Service limits, e.g. concurrent requests
36
Deployment Scenarios
37
ext*
XFS
ZFS
~/
~/scratch
Common endpoint configuration (GCSv4)
Data Transfer Node
POSIX
Connector
ext*
XFS
ZFS
GPFS
Lustre
~/projects
Common endpoint configuration (GCSv4)
Data Transfer Node
POSIX
Connector
~/
~/scratch
ext*
XFS
ZFS
GPFS
Lustre
~/projects
Multi-endpoint configuration (GCSv4)
Data Transfer Node
POSIX
Connector
Western
Digital
ActiveScale
Connector
~/archive
~/
~/scratch
~/vault
ext*
XFS
ZFS
GPFS
Lustre
~/projects
Multi-endpoint configuration (GCSv4)
41
Data Transfer Node
POSIX
Connector
Western
Digital
ActiveScale
Connector
~/archive
Amazon
S3
Bucket
Amazon S3
Connector
~/
~/scratch
Network paths
• Separate control and data interfaces
• "DataInterface =" option in globus-connect-server-
conf
• Common scenario: route data flows over Science
DMZ link
42
Dual-homed DTN – high speed data path
Data
Transfer
Node
GridFTP
Server
Science DMZ
Control
Channel
Data
Transfer
Node
GridFTP
Server
Data Channel
if0
if1
Internet2
path
Control
Channel
Dual-homed DTN – internal data path
Data
Transfer
Node
GridFTP
Server
Science DMZ
Control
Channel
Data
Transfer
Node
GridFTP
Server
Data Channel
if0
if1
LAN/
Intranet
path
Control
Channel
Firewall
if0
if1
Other Deployment Options
45
Encryption
• Requiring encryption on an endpoint
– User cannot override
– Useful for “sensitive” data
• Globus uses OpenSSL cipher stack as currently
configured on your DTN
• FIPS 140-2 compliance: ensure use of FIPS capable
OpenSSL libraries on DTN
www.openssl.org/docs/fips/UserGuide-2.0.pdf
46
Distributing Globus Connect Server components
• Globus Connect Server components
– globus-connect-server-io, -id, -web
• Default: -io, –id and –web on single server
• Common options
– Multiple –io servers for load balancing, failover, and
performance
– No -id server, e.g. third-party IdP
– -id on separate server, e.g. non-DTN nodes
– -web on either –id server or separate server for OAuth interface
47
ext*
XFS
ZFS
Distributing Globus Connect Server components
Data
Transfer
Node
OAuth
Server
GridFTP
Server
MyProxy
CA
Science DMZ
(ACL limited)
Port 2811
accepts inbound
connections
from Globus
Firewall
Setting up multiple –io servers
• Guidelines
– Use the same .conf file on all servers
– First install on the server running the –id component, then all others
• Install Globus Connect Server on all servers
• Edit .conf file on one of the servers and set [MyProxy] Server to the hostname
of the server you want the –id component installed on
• Copy Globus Connect Server configuration file to all servers
• Run globus-connect-server-setup on the server running the –id component
• Run globus-connect-server-setup on all other servers
• Repeat steps 2-5 as necessary to update configurations
49
Example: Two-node DTN
50
-id
-io
-io
On other DTN nodes:
/etc/globus-connect-server.conf
[Endpoint] Name = globus_dtn
[MyProxy] Server = 34.20.29.57
On “primary” DTN node (34.20.29.57):
/etc/globus-connect-server.conf
[Endpoint] Name = globus_dtn
[MyProxy] Server = 34.20.29.57
Open Discussion
51

More Related Content

What's hot

Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
Globus
 
Develop with linux containers and docker
Develop with linux containers and dockerDevelop with linux containers and docker
Develop with linux containers and docker
Fabio Fumarola
 
Dockerizing a multi-component Open Data app
Dockerizing a multi-component Open Data app  Dockerizing a multi-component Open Data app
Dockerizing a multi-component Open Data app
Stergios Tsiafoulis
 
Automating Research Data Workflows (GlobusWorld Tour - UCSD)
Automating Research Data Workflows (GlobusWorld Tour - UCSD)Automating Research Data Workflows (GlobusWorld Tour - UCSD)
Automating Research Data Workflows (GlobusWorld Tour - UCSD)
Globus
 
From Kubernetes to OpenStack in Sydney
From Kubernetes to OpenStack in SydneyFrom Kubernetes to OpenStack in Sydney
From Kubernetes to OpenStack in Sydney
SK Telecom
 
Docker 1.5
Docker 1.5Docker 1.5
Docker 1.5
rajdeep
 
wget, curl and scp
wget, curl and scpwget, curl and scp
wget, curl and scp
Gaurav Mishra
 
60 Admin Tips
60 Admin Tips60 Admin Tips
60 Admin Tips
Gabriella Davis
 
HTTP 프로토콜의 이해와 활용
HTTP 프로토콜의 이해와 활용HTTP 프로토콜의 이해와 활용
HTTP 프로토콜의 이해와 활용
SangJin Kang
 
Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)
DongHyeon Kim
 
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBayReal-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Altinity Ltd
 
Web Performance Optimization with HTTP/3
Web Performance Optimization with HTTP/3Web Performance Optimization with HTTP/3
Web Performance Optimization with HTTP/3
SangJin Kang
 
January OpenNTF Webinar - Backup your Domino Server - New Options in V12
January OpenNTF Webinar - Backup your Domino Server - New Options in V12January OpenNTF Webinar - Backup your Domino Server - New Options in V12
January OpenNTF Webinar - Backup your Domino Server - New Options in V12
Howard Greenberg
 
ClickHouse Keeper
ClickHouse KeeperClickHouse Keeper
ClickHouse Keeper
Altinity Ltd
 
Kubernetes 1.3 - Highlights
Kubernetes 1.3 - HighlightsKubernetes 1.3 - Highlights
Kubernetes 1.3 - Highlights
Matthew Barker
 
LibX 2.0
LibX 2.0LibX 2.0
LibX 2.0
eby
 
June OpenNTF Webinar - Domino V12 Certification Manager
June OpenNTF Webinar - Domino V12 Certification ManagerJune OpenNTF Webinar - Domino V12 Certification Manager
June OpenNTF Webinar - Domino V12 Certification Manager
Howard Greenberg
 
2 Linux Container and Docker
2 Linux Container and Docker2 Linux Container and Docker
2 Linux Container and Docker
Fabio Fumarola
 
Get your instance by name integration of nova, neutron and designate
Get your instance by name  integration of nova, neutron and designateGet your instance by name  integration of nova, neutron and designate
Get your instance by name integration of nova, neutron and designate
Miguel Lavalle
 
Jsonnet, terraform & packer
Jsonnet, terraform & packerJsonnet, terraform & packer
Jsonnet, terraform & packer
David Cunningham
 

What's hot (20)

Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
 
Develop with linux containers and docker
Develop with linux containers and dockerDevelop with linux containers and docker
Develop with linux containers and docker
 
Dockerizing a multi-component Open Data app
Dockerizing a multi-component Open Data app  Dockerizing a multi-component Open Data app
Dockerizing a multi-component Open Data app
 
Automating Research Data Workflows (GlobusWorld Tour - UCSD)
Automating Research Data Workflows (GlobusWorld Tour - UCSD)Automating Research Data Workflows (GlobusWorld Tour - UCSD)
Automating Research Data Workflows (GlobusWorld Tour - UCSD)
 
From Kubernetes to OpenStack in Sydney
From Kubernetes to OpenStack in SydneyFrom Kubernetes to OpenStack in Sydney
From Kubernetes to OpenStack in Sydney
 
Docker 1.5
Docker 1.5Docker 1.5
Docker 1.5
 
wget, curl and scp
wget, curl and scpwget, curl and scp
wget, curl and scp
 
60 Admin Tips
60 Admin Tips60 Admin Tips
60 Admin Tips
 
HTTP 프로토콜의 이해와 활용
HTTP 프로토콜의 이해와 활용HTTP 프로토콜의 이해와 활용
HTTP 프로토콜의 이해와 활용
 
Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)Kubernetes internals (Kubernetes 해부하기)
Kubernetes internals (Kubernetes 해부하기)
 
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBayReal-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
Real-time, Exactly-once Data Ingestion from Kafka to ClickHouse at eBay
 
Web Performance Optimization with HTTP/3
Web Performance Optimization with HTTP/3Web Performance Optimization with HTTP/3
Web Performance Optimization with HTTP/3
 
January OpenNTF Webinar - Backup your Domino Server - New Options in V12
January OpenNTF Webinar - Backup your Domino Server - New Options in V12January OpenNTF Webinar - Backup your Domino Server - New Options in V12
January OpenNTF Webinar - Backup your Domino Server - New Options in V12
 
ClickHouse Keeper
ClickHouse KeeperClickHouse Keeper
ClickHouse Keeper
 
Kubernetes 1.3 - Highlights
Kubernetes 1.3 - HighlightsKubernetes 1.3 - Highlights
Kubernetes 1.3 - Highlights
 
LibX 2.0
LibX 2.0LibX 2.0
LibX 2.0
 
June OpenNTF Webinar - Domino V12 Certification Manager
June OpenNTF Webinar - Domino V12 Certification ManagerJune OpenNTF Webinar - Domino V12 Certification Manager
June OpenNTF Webinar - Domino V12 Certification Manager
 
2 Linux Container and Docker
2 Linux Container and Docker2 Linux Container and Docker
2 Linux Container and Docker
 
Get your instance by name integration of nova, neutron and designate
Get your instance by name  integration of nova, neutron and designateGet your instance by name  integration of nova, neutron and designate
Get your instance by name integration of nova, neutron and designate
 
Jsonnet, terraform & packer
Jsonnet, terraform & packerJsonnet, terraform & packer
Jsonnet, terraform & packer
 

Similar to Tutorial: Introduction to Globus for System Administrators

Globus Endpoint Administration (GlobusWorld Tour - STFC)
Globus Endpoint Administration (GlobusWorld Tour - STFC)Globus Endpoint Administration (GlobusWorld Tour - STFC)
Globus Endpoint Administration (GlobusWorld Tour - STFC)
Globus
 
GlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System Administrators
Globus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
Globus
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
Globus
 
Advanced Globus System Administration Topics
Advanced Globus System Administration TopicsAdvanced Globus System Administration Topics
Advanced Globus System Administration Topics
Globus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
Globus
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
Globus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
Globus
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
Globus
 
GlobusWorld 2021 Tutorial: The Globus CLI, Platform and SDK
GlobusWorld 2021 Tutorial: The Globus CLI, Platform and SDKGlobusWorld 2021 Tutorial: The Globus CLI, Platform and SDK
GlobusWorld 2021 Tutorial: The Globus CLI, Platform and SDK
Globus
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
Globus
 
Data Publication and Discovery with Globus
Data Publication and Discovery with GlobusData Publication and Discovery with Globus
Data Publication and Discovery with Globus
Globus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
Globus
 
Connecting Your System to Globus (APS Workshop)
Connecting Your System to Globus (APS Workshop)Connecting Your System to Globus (APS Workshop)
Connecting Your System to Globus (APS Workshop)
Globus
 
Automating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus PlatformAutomating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus Platform
Globus
 
Globus Command Line Interface (APS Workshop)
Globus Command Line Interface (APS Workshop)Globus Command Line Interface (APS Workshop)
Globus Command Line Interface (APS Workshop)
Globus
 
Migrating to Globus Connect Server v5
Migrating to Globus Connect Server v5Migrating to Globus Connect Server v5
Migrating to Globus Connect Server v5
Globus
 
Automating Research Data Flows and an Introduction to the Globus Platform
Automating Research Data Flows and an Introduction to the Globus PlatformAutomating Research Data Flows and an Introduction to the Globus Platform
Automating Research Data Flows and an Introduction to the Globus Platform
Globus
 
Globus Endpoint Migration and Advanced Administration Topics
Globus Endpoint Migration and Advanced Administration TopicsGlobus Endpoint Migration and Advanced Administration Topics
Globus Endpoint Migration and Advanced Administration Topics
Globus
 
Jupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data ScienceJupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data Science
Globus
 

Similar to Tutorial: Introduction to Globus for System Administrators (20)

Globus Endpoint Administration (GlobusWorld Tour - STFC)
Globus Endpoint Administration (GlobusWorld Tour - STFC)Globus Endpoint Administration (GlobusWorld Tour - STFC)
Globus Endpoint Administration (GlobusWorld Tour - STFC)
 
GlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System AdministratorsGlobusWorld 2021 Tutorial: Globus for System Administrators
GlobusWorld 2021 Tutorial: Globus for System Administrators
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
 
Advanced Globus System Administration Topics
Advanced Globus System Administration TopicsAdvanced Globus System Administration Topics
Advanced Globus System Administration Topics
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
 
GlobusWorld 2021 Tutorial: The Globus CLI, Platform and SDK
GlobusWorld 2021 Tutorial: The Globus CLI, Platform and SDKGlobusWorld 2021 Tutorial: The Globus CLI, Platform and SDK
GlobusWorld 2021 Tutorial: The Globus CLI, Platform and SDK
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
 
Data Publication and Discovery with Globus
Data Publication and Discovery with GlobusData Publication and Discovery with Globus
Data Publication and Discovery with Globus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 
Connecting Your System to Globus (APS Workshop)
Connecting Your System to Globus (APS Workshop)Connecting Your System to Globus (APS Workshop)
Connecting Your System to Globus (APS Workshop)
 
Automating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus PlatformAutomating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus Platform
 
Globus Command Line Interface (APS Workshop)
Globus Command Line Interface (APS Workshop)Globus Command Line Interface (APS Workshop)
Globus Command Line Interface (APS Workshop)
 
Migrating to Globus Connect Server v5
Migrating to Globus Connect Server v5Migrating to Globus Connect Server v5
Migrating to Globus Connect Server v5
 
Automating Research Data Flows and an Introduction to the Globus Platform
Automating Research Data Flows and an Introduction to the Globus PlatformAutomating Research Data Flows and an Introduction to the Globus Platform
Automating Research Data Flows and an Introduction to the Globus Platform
 
Globus Endpoint Migration and Advanced Administration Topics
Globus Endpoint Migration and Advanced Administration TopicsGlobus Endpoint Migration and Advanced Administration Topics
Globus Endpoint Migration and Advanced Administration Topics
 
Jupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data ScienceJupyter + Globus: The Foundation for Interactive Data Science
Jupyter + Globus: The Foundation for Interactive Data Science
 

More from Globus

Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus
 
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Globus
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
Globus
 
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus
 
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Globus
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Globus
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
Globus
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
Globus
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
Globus
 
How to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good PracticesHow to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good Practices
Globus
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Globus
 
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Globus
 
The Department of Energy's Integrated Research Infrastructure (IRI)
The Department of Energy's Integrated Research Infrastructure (IRI)The Department of Energy's Integrated Research Infrastructure (IRI)
The Department of Energy's Integrated Research Infrastructure (IRI)
Globus
 
GlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote sessionGlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote session
Globus
 
Enhancing Performance with Globus and the Science DMZ
Enhancing Performance with Globus and the Science DMZEnhancing Performance with Globus and the Science DMZ
Enhancing Performance with Globus and the Science DMZ
Globus
 
Extending Globus into a Site-wide Automated Data Infrastructure.pdf
Extending Globus into a Site-wide Automated Data Infrastructure.pdfExtending Globus into a Site-wide Automated Data Infrastructure.pdf
Extending Globus into a Site-wide Automated Data Infrastructure.pdf
Globus
 
Globus at the United States Geological Survey
Globus at the United States Geological SurveyGlobus at the United States Geological Survey
Globus at the United States Geological Survey
Globus
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Globus
 
Globus Compute with Integrated Research Infrastructure (IRI) workflows
Globus Compute with Integrated Research Infrastructure (IRI) workflowsGlobus Compute with Integrated Research Infrastructure (IRI) workflows
Globus Compute with Integrated Research Infrastructure (IRI) workflows
Globus
 
Reactive Documents and Computational Pipelines - Bridging the Gap
Reactive Documents and Computational Pipelines - Bridging the GapReactive Documents and Computational Pipelines - Bridging the Gap
Reactive Documents and Computational Pipelines - Bridging the Gap
Globus
 

More from Globus (20)

Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
 
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
 
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024
 
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
 
How to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good PracticesHow to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good Practices
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
 
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...
 
The Department of Energy's Integrated Research Infrastructure (IRI)
The Department of Energy's Integrated Research Infrastructure (IRI)The Department of Energy's Integrated Research Infrastructure (IRI)
The Department of Energy's Integrated Research Infrastructure (IRI)
 
GlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote sessionGlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote session
 
Enhancing Performance with Globus and the Science DMZ
Enhancing Performance with Globus and the Science DMZEnhancing Performance with Globus and the Science DMZ
Enhancing Performance with Globus and the Science DMZ
 
Extending Globus into a Site-wide Automated Data Infrastructure.pdf
Extending Globus into a Site-wide Automated Data Infrastructure.pdfExtending Globus into a Site-wide Automated Data Infrastructure.pdf
Extending Globus into a Site-wide Automated Data Infrastructure.pdf
 
Globus at the United States Geological Survey
Globus at the United States Geological SurveyGlobus at the United States Geological Survey
Globus at the United States Geological Survey
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
 
Globus Compute with Integrated Research Infrastructure (IRI) workflows
Globus Compute with Integrated Research Infrastructure (IRI) workflowsGlobus Compute with Integrated Research Infrastructure (IRI) workflows
Globus Compute with Integrated Research Infrastructure (IRI) workflows
 
Reactive Documents and Computational Pipelines - Bridging the Gap
Reactive Documents and Computational Pipelines - Bridging the GapReactive Documents and Computational Pipelines - Bridging the Gap
Reactive Documents and Computational Pipelines - Bridging the Gap
 

Recently uploaded

Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
Dinusha Kumarasiri
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 

Recently uploaded (20)

Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 

Tutorial: Introduction to Globus for System Administrators

  • 1. Globus for System Administrators Vas Vasiliadis vas@uchicago.edu May 2, 2019
  • 2. Globus Connect Server 2 • Makes your storage accessible via Globus • Multi-user server, installed and managed by sysadmin docs.globus.org/globus-connect-server-installation-guide/ Local system users Local Storage System (HPC cluster, NAS, …) Globus Connect Server MyProxy CA GridFTP Server OAuth Server DTN • Default access for all local accounts • Native packaging Linux: DEB, RPM
  • 3. Globus Connect Server 3 Local system users Local Storage System (HPC cluster, NAS, …) Globus Connect Server MyProxy CA GridFTP Server OAuth Server DTN Non-POSIX Connectors POSIX-compliant Connector server
  • 4. IBM Spectrum Scale Current Planned Storage Connectors - globus.org/connectors
  • 5. Globus Connect Server v5 Milestones v5.0: Google Drive v5.1: POSIX guest collections, HTTPS v5.x+: v4 feature parity+v5.x: … • Multi DTN support Additional connectors • Custom IdPs • … Other features v5.2: High assurance v5.3: Version Unification
  • 6. By default, assume you should use GCS v4 Which version of Globus Connect Server do I use?
  • 7. Which version of Globus Connect Server do I use? Are you a Globus subscriber? Do you have a High Assurance or BAA subscription (and plan to set up a high assurance endpoint)? Use GCS v4 Use GCS v5.3 No Yes No Yes Would you like to support multiple storage system types on one endpoint? Do you want to enable HTTPS access to your storage? Do you plan to use the S3, Box, Google Drive or Ceph connectors? No No No Yes Yes Yes
  • 8. Creating a Globus endpoint on your server • In this example, Server = Amazon EC2 instance • Installation and configuration of Globus Connect Server requires a Globus ID • Go to globusid.org • Click “create a Globus ID” – Optional: associate it with your Globus account 8
  • 9. What we are going to do: Install Globus Connect Server • Access server as user “campusadmin” • Update repo • Install package • Setup Globus Connect Server Server (AWS EC2) ssh ec2-22-23-24-25 Log into Globus Transfer a file 1 2 3 Access the newly created endpoint (as user ‘researcher’) 4 9
  • 10. Access your server • Get the IP address for your EC2 server (bit.ly/ec2ip) • Log in as user ‘campusadmin’ ssh campusadmin@<EC2_instance_IP_address> • Please sudo su before continuing – User ‘campusadmin’ has passwordless sudo privileges 10
  • 11. Install Globus Connect Server $ sudo su $ curl –LOs http://downloads.globus.org/toolkit/globus-connect- server/globus-connect-server-repo_latest_all.deb $ dpkg –i globus-connect-server-repo_latest_all.deb $ apt-get update $ apt-get -y install globus-connect-server $ globus-connect-server-setup You have a working Globus endpoint! Use your Globus ID username and password when prompted
  • 12. Access the Globus endpoint • Go to Manage Data à Transfer Files • Access the endpoint you just created – Search for your EC2 host name in the Endpoint field – Log in as “researcher”; you will see the user’s home directory • Transfer files between a test endpoint (e.g. ESnet read-only) and your EC2 endpoint 12
  • 13. Globus accounts and endpoint access • Globus account: Primary identity (+ Linked Identities) • Endpoint initially accessible by creator • Endpoint not visible? – Primary identity is your institutional ID? – Link your Globus ID!
  • 15. Endpoint configuration • On the Globus service: app.globus.org/endpoints • On your DTN: /etc/globus-connect-server.conf – Standard .ini format: [Section] Option = Value – To enable changes run globus-connect-server-setup – “Rinse and repeat” 15
  • 16. Common configuration options • Endpoints page – Display Name – Visibility – Encryption • DTN configuration file – RestrictPaths – Sharing – IdentityMethod (CILogon, Oauth) – SharingRestrictPaths
  • 17. Exercise: Make your endpoint visible • Edit endpoint attributes – Change the name to something useful, e.g. <your_name> EC2 Endpoint – For the “Visible To” attribute select “Public - Visible to all users” • Find your neighbor’s endpoint – Thanks to our superb security …you can access it too J 17
  • 18. Path Restriction • Default configuration: – All paths allowed, access control handled by the OS • Use RestrictPaths to customize – Specifies a comma separated list of full paths that clients may access – Each path may be prefixed by R (read) and/or W (write), or N (none) to explicitly deny access to a path – '~’ for authenticated user’s home directory, and * may be used for simple wildcard matching. • e.g. Full access to home directory, read access to /data: – RestrictPaths = RW~,R/data • e.g. Full access to home directory, deny hidden files: – RestrictPaths = RW~,N~/.* 18
  • 19. Exercise: Restrict access • Set RestrictPaths=RW~,N~/archive • Run globus-connect-server-setup • Access your endpoint as ‘researcher’ • What’s changed? 19
  • 20. • In config file, set Sharing=True • Run globus-connect-server-setup • Flag endpoint as “managed” (in web app or via CLI) * Note: Creation of shared endpoints requires a Globus subscription for the managed endpoint Enabling sharing on an endpoint 20
  • 21. Limit sharing to specific accounts • SharingUsersAllow = • SharingGroupsAllow = • SharingUsersDeny = • SharingGroupsDeny = 21
  • 22. Sharing Path Restriction • Restrict paths where users can create shared endpoints • Use SharingRestrictPaths to customize – Same syntax as RestrictPaths • e.g. Full access to home directory, deny hidden files: – SharingRestrictPaths = RW~,N~/.* • e.g. Full access to public folder under home directory: – SharingRestrictPaths = RW~/public • e.g. Full access to /proj, read access to /scratch: – SharingRestrictPaths = RW/proj,R/scratch 22
  • 24. Ports needed for Globus • Inbound: 2811 (control channel) • Inbound: 7512 (MyProxy), 443 (OAuth) • Inbound: 50000-51000 (data channel) • If restricting outbound connections, allow connections on: – 80, 2223 (used during install/config) – 50000-51000 (GridFTP data channel) 24
  • 27. Single Sign-On with InCommon/CILogon • Your Shibboleth server must release R&S attributes to CILogon—especially the ePPN attribute • Local account must match institutional ID (InCommon ID) – Test by creating a local user with same name • In /etc/globus-connect-server.conf set: AuthorizationMethod = CILogon CILogonIdentityProvider = <institution_listed_in_CILogon_IdP_list> 27
  • 29. Subscription configuration • Subscription manager – Create/upgrade managed endpoints – Requires Globus ID linked to Globus account • Management console permissions – Independent of subscription manager – Map managed endpoint to Globus ID • Globus Plus group – Subscription Manager is admin – Can grant admin rights to other members 29
  • 30. Creating managed endpoints • Required for sharing, management console, reporting, … • Convert existing endpoint to managed via CLI (or web): globus endpoint update --managed <endpt_uuid> • Must be run by subscription manager • Important: Re-run endpoint update after deleting/re- creating endpoint 30
  • 31. Monitoring and managing Globus endpoint activity 31
  • 32. Management console • Monitor all transfers • Pause/resume specific transfers • Add pause conditions with various options • Resume specific tasks overriding pause conditions • Cancel tasks • View sharing ACLs 32
  • 33. Endpoint Roles • Administrator: define endpoint and roles • Access Manager: manage permissions • Activity Manager: perform control tasks • Activity Monitor: view activity 33
  • 36. Balance: performance - reliability • Network use parameters: concurrency, parallelism • Maximum, Preferred values for each • Transfer considers source and destination endpoint settings min( max(preferred src, preferred dest), max src, max dest ) • Service limits, e.g. concurrent requests 36
  • 38. ext* XFS ZFS ~/ ~/scratch Common endpoint configuration (GCSv4) Data Transfer Node POSIX Connector
  • 39. ext* XFS ZFS GPFS Lustre ~/projects Common endpoint configuration (GCSv4) Data Transfer Node POSIX Connector ~/ ~/scratch
  • 40. ext* XFS ZFS GPFS Lustre ~/projects Multi-endpoint configuration (GCSv4) Data Transfer Node POSIX Connector Western Digital ActiveScale Connector ~/archive ~/ ~/scratch
  • 41. ~/vault ext* XFS ZFS GPFS Lustre ~/projects Multi-endpoint configuration (GCSv4) 41 Data Transfer Node POSIX Connector Western Digital ActiveScale Connector ~/archive Amazon S3 Bucket Amazon S3 Connector ~/ ~/scratch
  • 42. Network paths • Separate control and data interfaces • "DataInterface =" option in globus-connect-server- conf • Common scenario: route data flows over Science DMZ link 42
  • 43. Dual-homed DTN – high speed data path Data Transfer Node GridFTP Server Science DMZ Control Channel Data Transfer Node GridFTP Server Data Channel if0 if1 Internet2 path Control Channel
  • 44. Dual-homed DTN – internal data path Data Transfer Node GridFTP Server Science DMZ Control Channel Data Transfer Node GridFTP Server Data Channel if0 if1 LAN/ Intranet path Control Channel Firewall if0 if1
  • 46. Encryption • Requiring encryption on an endpoint – User cannot override – Useful for “sensitive” data • Globus uses OpenSSL cipher stack as currently configured on your DTN • FIPS 140-2 compliance: ensure use of FIPS capable OpenSSL libraries on DTN www.openssl.org/docs/fips/UserGuide-2.0.pdf 46
  • 47. Distributing Globus Connect Server components • Globus Connect Server components – globus-connect-server-io, -id, -web • Default: -io, –id and –web on single server • Common options – Multiple –io servers for load balancing, failover, and performance – No -id server, e.g. third-party IdP – -id on separate server, e.g. non-DTN nodes – -web on either –id server or separate server for OAuth interface 47
  • 48. ext* XFS ZFS Distributing Globus Connect Server components Data Transfer Node OAuth Server GridFTP Server MyProxy CA Science DMZ (ACL limited) Port 2811 accepts inbound connections from Globus Firewall
  • 49. Setting up multiple –io servers • Guidelines – Use the same .conf file on all servers – First install on the server running the –id component, then all others • Install Globus Connect Server on all servers • Edit .conf file on one of the servers and set [MyProxy] Server to the hostname of the server you want the –id component installed on • Copy Globus Connect Server configuration file to all servers • Run globus-connect-server-setup on the server running the –id component • Run globus-connect-server-setup on all other servers • Repeat steps 2-5 as necessary to update configurations 49
  • 50. Example: Two-node DTN 50 -id -io -io On other DTN nodes: /etc/globus-connect-server.conf [Endpoint] Name = globus_dtn [MyProxy] Server = 34.20.29.57 On “primary” DTN node (34.20.29.57): /etc/globus-connect-server.conf [Endpoint] Name = globus_dtn [MyProxy] Server = 34.20.29.57