GlobusWORLD 2012Innovating Research Data ManagementIan Foster – April 2012
The data delugewww.globustoolkit.org   www.globusonline.org
Run a research project from a coffee shop?www.globustoolkit.org                3                www.globusonline.org
Towards “research IT as a service”     • Dark Energy Survey      •   SBGrid structural biology consortium     • Galaxy gen...
Let’s rethink how we provide research IT     Accelerate discovery and innovation worldwide by     providing research IT as...
2011: Grid meets Cloud         Globus Toolkit                  Globus Online                 Build the Grid            Use...
Globus Online – The first 18 months     • >4,500 registered users; adding ~300/month     • >3.5 PB moved; averaging ~50 TB...
Globus Online – The first 18 months     • 514 active endpoints (year-to-date) including          62 at Tier 1 research ins...
Our user communitieswww.globustoolkit.org             9        www.globusonline.org
10www.globustoolkit.org   www.globusonline.org
2012: Move. Store. Collaborate.                                                                  globus online            ...
Towards “research IT as a service”                  Research Data Management-as-a-Service                    Globus     Gl...
Globus Storage: For when you want to …  • Place your data    where you want  • Access it from                             ...
Globus Collaborate: For when you want to     Join with a few or many people to:     • Share documents     • Track tasks   ...
Globus Integrate: For when you want to…     Write programs that access/manage user     identities, profiles, groups, resou...
Earth System Grid – Portal integration • Outsource data transfer to Globus       – Data download to user machine         f...
www.globustoolkit.org   www.globusonline.org
Let’s take a look at what’s coming…  Traumatic Brain Injury (TBI)                                Diffusion Tensor Imaging ...
Let’s take a look at what’s coming…                                                Globus Connect                         ...
Globus Toolkit updateGT Releases 2011-2012     5/4/11 GT 5.1.0    5/18/11 GT 5.0.4      8/2/11 GT 5.1.1                   ...
Globus Toolkit update – GridFTP    • GT 5.2.0         – Support for DCSC command              • Allows client to specify c...
Delivery plans                   Globus Transfer           Globus Storage           Globus Collaborate                • Ge...
Delivery plans     • Globus Storage           – Current: Early access – contact us if interested           – Open beta pla...
Premium offerings     Motivated by sustainability: resource cost     recovery, user support, future development     • Glob...
Premium offerings     Motivated by sustainability: resource cost     recovery, user support, future development     • Glob...
The Globus yeamBryce Allen      Rachana                               Lisa                 Ananthakris                    ...
Thank you!www.globustoolkit.org       27       www.globusonline.org
Upcoming SlideShare
Loading in …5
×

GlobusWorld 2012 Foster Keynote

1,859 views
1,777 views

Published on

My introductory talk at GlobusWorld 2012, in which I review the progress made with Globus Online during the last year, and introduce our new Globus Storage, Globus Collaborate, and Globus Integrate services. A great demo showed it all working.

Published in: Technology, Health & Medicine
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,859
On SlideShare
0
From Embeds
0
Number of Embeds
245
Actions
Shares
0
Downloads
24
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Genome (x10,000 in 6 years!), astronomy, social sciences, HPC …Big projects, e.g., LHC, have things under controlWhat about small and medium labs? They are struggling
  • Small and medium businesses deal with these challenges by outsourcing many business functions to third-party “cloud” providers.Last year we introduced the seemingly farfetched notion that one could run a research project from a coffee shop…
  • We have been investigating over the past few years just what those services may be.
  • Not (particularly) computing as a serviceBut the IT functions that researchers need to functionInclude collaboration as a service
  • An example of integrating Globus services, in this case Globus Transfer, using Globus NexusMention branding is/will be available for every Globus service100 GB/sec
  • Moving beyond Transfer service… introduce planned services hereAlso introduce the new naming hereToday: Globus Online Globus Transfer; moving to Globus (+ Service Name)
  • Put the new services in context…
  • Introduce the Globus Nexus here…
  • GlobusWorld 2012 Foster Keynote

    1. 1. GlobusWORLD 2012Innovating Research Data ManagementIan Foster – April 2012
    2. 2. The data delugewww.globustoolkit.org www.globusonline.org
    3. 3. Run a research project from a coffee shop?www.globustoolkit.org 3 www.globusonline.org
    4. 4. Towards “research IT as a service” • Dark Energy Survey • SBGrid structural biology consortium • Galaxy genomics • NCAR climate data applications • LIGO observatory • Land use change; economicswww.globustoolkit.org www.globusonline.org
    5. 5. Let’s rethink how we provide research IT Accelerate discovery and innovation worldwide by providing research IT as a service Leverage the cloud to • provide millions of researchers with unprecedented access to powerful tools; • enable a massive shortening of cycle times in time-consuming research processes; and • reduce research IT costs dramatically via economies of scalewww.globustoolkit.org www.globusonline.org
    6. 6. 2011: Grid meets Cloud Globus Toolkit Globus Online Build the Grid Use the Grid Components for building Reliable file transfer custom grid solutions Software-as-a-Service globustoolkit.org globusonline.orgwww.globustoolkit.org 6 www.globusonline.org
    7. 7. Globus Online – The first 18 months • >4,500 registered users; adding ~300/month • >3.5 PB moved; averaging ~50 TB/week Data Transferred (TB/week) Support Requests Received 200 250 200 150 150 100 100 50 50 0 0 Jan 2011 Feb 2012 Dec 2010 Jan 2011www.globustoolkit.org 7 www.globusonline.org
    8. 8. Globus Online – The first 18 months • 514 active endpoints (year-to-date) including 62 at Tier 1 research institutions • >390,000,000 files moved • >100,000 transfer requests • 99.9% uptime in 2012 • 18 identity providers supported, using multiple protocols: X.509, MyProxy OAuth, OpenIDwww.globustoolkit.org 8 www.globusonline.org
    9. 9. Our user communitieswww.globustoolkit.org 9 www.globusonline.org
    10. 10. 10www.globustoolkit.org www.globusonline.org
    11. 11. 2012: Move. Store. Collaborate. globus online Globus Globus Globus Transfer Storage Collaborate Globus Integrate Globus Connect Multi User Globus Connect Globus Toolkit Globus Nexuswww.globustoolkit.org 11 www.globusonline.org
    12. 12. Towards “research IT as a service” Research Data Management-as-a-Service Globus Globus Globus Globus …SaaS Transfer Storage Collaborate Catalog Globus Integrate …PaaSwww.globustoolkit.org www.globusonline.org
    13. 13. Globus Storage: For when you want to … • Place your data where you want • Access it from Globus Transfer, HTTP/REST, Desktop sync anywhere via different protocols • Update it, version it, Globus Storage and take snapshots volume • Share versions with who you want • Synchronize among Commercial National Campus storage service research computin locations provider center g centerwww.globustoolkit.org www.globusonline.org
    14. 14. Globus Collaborate: For when you want to Join with a few or many people to: • Share documents • Track tasks • Send email • Share data • Do whatever With: • Common groups • Delegated managementwww.globustoolkit.org www.globusonline.org
    15. 15. Globus Integrate: For when you want to… Write programs that access/manage user identities, profiles, groups, resources—and data … Globus Globus Globus Transfer Storage Collaborate Globus Integrate Globus Connect Multi User Globus Connect Globus Toolkit Globus Nexus … via REST APIs and command line programswww.globustoolkit.org www.globusonline.org
    16. 16. Earth System Grid – Portal integration • Outsource data transfer to Globus – Data download to user machine from search – Data transfer to another server by user – Replication of data between sites by administrator • No ESGF client software neededwww.globustoolkit.org 16 www.globusonline.org
    17. 17. www.globustoolkit.org www.globusonline.org
    18. 18. Let’s take a look at what’s coming… Traumatic Brain Injury (TBI) Diffusion Tensor Imaging (DTI)www.globustoolkit.org 18 www.globusonline.org
    19. 19. Let’s take a look at what’s coming… Globus Connect Bryce Move DTI results to PADS Bryce’s laptop Compute DTI Group Cluster - Kyle - Bryce Globus Storage Globus Transfer Create snapshot to Copy TBI data to share with group compute cluster Globus Nexus Globus Transfer Add Bryce to TBI Move DTI results collaboration to shared volume Globus Collaborate Publish DTI data to TBI web site Amazon S3 Globus Storage Create volume and share with TBI group SDSC UChicago CloudKyle “TBI” Object Globus Connect volume Store Move MRI files to Cornell TBI shared volume Red Cloudwww.globustoolkit.org 19 www.globusonline.org
    20. 20. Globus Toolkit updateGT Releases 2011-2012 5/4/11 GT 5.1.0 5/18/11 GT 5.0.4 8/2/11 GT 5.1.1 Highlights 11/16/11 GT 5.1.3 (5.2 beta) • Native packaging12/15/11 GT 5.2.0 • GridFTP: DCSC support 3/8/12 GT 5.0.5 • New GRAM5 version; Likely last 5.0.x release in use on OSG 4/5/12 GT 5.2.1rc1 5.2.1 final planned for Aprilwww.globustoolkit.org 20 www.globusonline.org
    21. 21. Globus Toolkit update – GridFTP • GT 5.2.0 – Support for DCSC command • Allows client to specify credentials used to secure data channel connection • Utilized by Globus Transfer for seamless data movement across multiple security domains – Server administrators may now restrict client access to a set of paths • Coming in 5.2.1 – MLSC – stream directory listings over control channel • Enables simple (control channel-only) implementation of Globus Storage ftp server • Can help directory listings for gridftp servers behindwww.globustoolkit.org restrictive firewalls 21 www.globusonline.org
    22. 22. Delivery plans Globus Transfer Globus Storage Globus Collaborate • Generally available • Early release • Initial projects at • Service and Web UI • Generally available in UChicago enhancements Fall 2012 • Early release sometime continue before year-end 2012 Globus Integrate Globus Connect Multi User (GA) • Transfer API available • APIs for Storage, Collaborate planned after app release Globus Connect (GA) Globus Toolkit (GA V5.2) Globus Nexus (Alpha)www.globustoolkit.org www.globusonline.org
    23. 23. Delivery plans • Globus Storage – Current: Early access – contact us if interested – Open beta planned in late Spring; GA in late Summer – V1.0: UChicago and Amazon S3 object stores • Globus Collaborate – Current: Pre-release with two UChicago/CI groups – First release planned around year-endwww.globustoolkit.org 23 www.globusonline.org
    24. 24. Premium offerings Motivated by sustainability: resource cost recovery, user support, future development • Globus Transfer plans (resource providers) – www.globusonline.org/premium-plans/ • Globus Storage premium plans – Plans variables include: size, object store type, durability, support level, personalization • Globus Collaborate premium plans – Plans variables include: branding capabilities, support level, integration (e.g. identity)www.globustoolkit.org 24 www.globusonline.org
    25. 25. Premium offerings Motivated by sustainability: resource cost recovery, user support, future development • Globus Transfer plans (resource providers) – www.globusonline.org/premium-plans/ • Globus Storage premium plans – Plans variables include: size, object store type, durability, support level, personalization • Globus Collaborate premium plans – Plans variables include: branding capabilities, support level, integration (e.g. identity)www.globustoolkit.org 25 www.globusonline.org
    26. 26. The Globus yeamBryce Allen Rachana Lisa Ananthakris Childers hnanMartin Tom Howe Raj Jack Kordas LukaszFeller Kettimuthu LacinskiLee Liming JP Navarro Karl Pickett Andrew Peter Zich Zichwww.globustoolkit.org 26 www.globusonline.org
    27. 27. Thank you!www.globustoolkit.org 27 www.globusonline.org

    ×