Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Globus: Beyond File Transfer


Published on

This presentation was given at the GlobusWorld 2020 Virtual Conference, by Greg Nawrocki from the University of Chicago.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Globus: Beyond File Transfer

  1. 1. Globus - Beyond File Transfer A Globus Introduction and more! Greg Nawrocki
  2. 2. Research data management today How do we... ...move? ...share? ...reproduce? Index?
  3. 3. 3 Globus delivers… Fast and reliable big data transfer, sharing, and platform services… …directly from your own storage systems… ...via software-as-a-service using existing identities with the overarching goal of...
  4. 4. 4 Research Computing HPC Desktop Workstations Mass Storage Instruments Personal Resources Public Cloud National Resources Unifying access to data across tiers
  5. 5. Globus Connectors ActiveScale Object Storage Planned
  6. 6. Public / private cloud stores External campus storage EC2 Project repositories, replication stores Public repositories Share with collaborators/community
  7. 7. Analysis store Next-Gen Sequencer MRI Advanced Light Source Personal system Remote visualization Light Sheet Microscope High-durability, low-cost store Manage data from instruments Cryo-EM
  8. 8. Use(r)-appropriate interfaces 8 GET /endpoint/go%23ep1 PUT /endpoint/vas#my_endpt 200 OK X-Transfer-API-Version: 0.10 Content-Type: application/json … Globus service Web CLI Rest API
  9. 9. Globus SaaS / PaaS: Research data lifecycle Researcher initiates transfer request; or requested automatically by script, science gateway 1 Instrument Compute Facility Globus transfers files reliably, securely 2 Globus controls access to shared files on existing storage; no need to move files to cloud storage! 4 Researcher selects files to share, selects user or group, and sets access permissions 3 Collaborator logs in to Globus and accesses shared files; no local account required; download via Globus 5 Automating research workflows and ensuring those that need access to the data have it. 8 Personal Computer Transfer Share • Use a Web browser or platform services • Access any storage • Use an existing identity Build The Globus Command Line Interface, API sets, and Python SDK provide a platform… 6 … for building science gateways, portals and publication services. 7
  10. 10. Conceptual architecture: Hybrid SaaS DATA Channel CONTROL Channel Source Endpoint Destination Endpoint Subscriber owned and administered storage system Globus “client” software No data relay or staging via Globus Subscriber Control Domain Globus Control Domain Single, globally accessible multi-tenant service User accessing the Globus Web App via a browser
  11. 11. Conceptual architecture: Sharing Managed Endpoint Subscriber Control Domain Globus Control Domain Globus managed ”overlay” permissions Shared Endpoint DATA Channel CONTROL Channel Subscriber managed filesystem permissions External User Control Domain
  12. 12. …makes your storage system a Globus endpoint
  13. 13. Endpoints (Collections) • Storage abstraction – All transfers happen between two endpoints – Globus Connect instantiates endpoints • Collection ~= Endpoint • Test / Demo Endpoints – Globus Tutorial Endpoint 1 – Globus Tutorial Endpoint 2 – ESnet Test Endpoints o Contain file samples of various sizes • Globus Connect Personal – Now your laptop is an endpoint – 14
  14. 14. Globus Connect Personal • Installers do not require admin access • Zero configuration; auto updating • Handles NATs • Installs in seconds – easy to delete - I’ll prove it!
  15. 15. Almost demo time… • How do I get a Globus account? – A Globus Account is o A Primary Identity o Possible Linked Identities – Your existing institutional identity may already work – Linking / Managing Identities – Consents • But I don’t have any Endpoints (Collections)! – Globus Connect Personal – Globus Tutorial Endpoint 1 – Globus Tutorial Endpoint 2 – ESnet Test Endpoints o Contain file samples of various sizes 16
  16. 16. Demo time! Identities and Accounts Transfer Sharing Transfer Details Bookmarks The Console The Hamburger Menu The Activity Monitor Groups Roles Responsive Interface
  17. 17. Globus Command Line Interface Open source, uses Python SDK globus-cli
  18. 18. Globus Auth API (Group Management) … GlobusTransferAPI GlobusConnect Data Discovery File Sharing File Transfer & Replication Globus Platform-as-a-Service Use existing institutional ID systems in external web applications Integrate file transfer and sharing capabilities into scientific web apps, portals, gateways, etc...
  19. 19. A bit of Globus history U . S . D E P A R T M E N T O F ENERGY
  20. 20. Globus sustainability model • Standard Subscription – Shared endpoints – Management console – Usage reporting – Priority support – Application integration – HTTPS support – Branded Web Site • Premium Storage Connectors • Alternate Identity Provider (InCommon is standard) 21
  21. 21. The path to sustainability
  22. 22. Globus by the numbers... 6764 active shared endpoints 115 subscribers 887+ PB moved 23,818 active personal endpoints 94 billion files processed 1866 active server endpoints 80 countries where Globus is used 2.9 PB largest single transfer to date 99.9% availability 745 identity providers 1960 most shared endpoints at a single institution 112,105 total users
  23. 23. Manage Protected Data 24 Higher assurance levels for HIPAA and other regulated data • Support for protected data such as health related information • Share data with collaborators while meeting compliance requirements • Includes BAA option
  24. 24. Globus for high assurance data management • Restricted data handling – PII (Personally identifiable information) – Controlled Unclassified Information – PHI (Protected Health Information) • University of Chicago security controls – NIST 800-53 Low – Superset of 800-171 Low • Business Associate Agreements (BAA) will be between University of Chicago and our subscribers – University of Chicago has a BAA with Amazon
  25. 25. High Assurance features • Additional authentication assurance – Per storage gateway policy on frequency of authentication with specific identity for access to data (timeout) – Ensure that user authenticates with the specific identity that gives them access within session (decoupling linked identities) • Session/device isolation – Authentication context is per application, per session (~browser session) • Enforces encryption of all user data in transit • Audit logging
  26. 26. Support resources • Globus documentation: • Helpdesk and issue escalation: • Mailing Lists – • Customer engagement team • Globus professional services team – Assist with portal/gateway/app architecture and design – Develop custom applications that leverage the Globus platform – Advise on customized deployment and integration scenarios
  27. 27. Globus on your Campus • Webinars • Programs – Helping you evangelize Globus within your institution. • Professional Services • Globus World Tour – Taking the show on the road. 28