Services for Science v2 (APAN26)

  • 661 views
Uploaded on

Talk given at APAN26 in beautiful Queenstown, New Zealand. Updates my INGRID talk with new material on caBIG--including some nice slides provided by Ravi Madduri.

Talk given at APAN26 in beautiful Queenstown, New Zealand. Updates my INGRID talk with new material on caBIG--including some nice slides provided by Ravi Madduri.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
661
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
6
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Services for Science Ian Foster Computation Institute Argonne National Lab & University of Chicago
  • 2. Thanks!
    • DOE Office of Science
    • NSF Office of Cyberinfrastructure
    • National Institutes of Health
    • Colleagues at Argonne, U.Chicago, USC/ISI, OSU, Manchester, and elsewhere
  • 3. Scientific Communication, ~1600 Brahe Kepler
  • 4. 1980
  • 5. Scientific Communication, ~2000 Data Archives Gateway Figure: S. G. Djorgovski Service-Oriented Science User Analysis tools Discovery tools
  • 6. Application Scenario Location A Microarray, Protein, Image data Location B Microarray, Protein, Image data Location C Microarray, Protein, Image data Location C Image Analysis Location D Image Analysis Microarray and protein databases at other institutions Different database systems, data representations, security Different program invocation, remote access, data transfer
  • 7. caBIG: sharing of infrastructure, applications, and data. Data Integration! Services & Cancer Biology Globus
  • 8. Service-Oriented Science
    • People create services (data, code, instr.) …
    • which I discover (& decide whether to use) …
    • & compose to create a new function ...
    • & then publish as a new service.
    •  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
    •  I hope that this “someone else” can manage security, reliability, scalability, …
    ! ! “ Service-Oriented Science”, Science , 2005
  • 9. Creating Services
    • People create services (data, code, instr.) …
    • which I discover (& decide whether to use) …
    • & compose to create a new function ...
    • & then publish as a new service.
    •  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
    •  I hope that this “someone else” can manage security, reliability, scalability, …
    ! ! “ Service-Oriented Science”, Science , 2005
  • 10. Anatomy of a Service op1 opN (meta)data Implementation(s) Clients Registry Management Clients … Service Service Attribute Authority Attribute Authority Persistence
  • 11. Creating Services (~2005)
    • “ This full-day tutorial provides an introduction to programming Java services with the latest version of the Globus Toolkit version 4 (GT4). The tutorial teaches how to build a Java Service that makes use of GT4 mechanisms for state management, security, registry and related topics.”
  • 12. Creating Services in 2008 Introduce and gRAVI
    • Introduce
      • Define service
      • Create skeleton
      • Discover types
      • Add operations
      • Configure security
    • Grid R emote A pplication V irtualization Infrastructure
      • Wrap executables
    Index service Repository Service Introduce Container Ohio State University and Argonne/U.Chicago Appln Service Create Store Advertize Discover Invoke; get results Transfer GAR Deploy Globus
  • 13. Demonstration: Creating Services Introduce + gRAVI Shannon Hastings Scott Oster David Ervin Stephen Langella Kyle Chard Ravi Madduri
  • 14. Center for Enabling Distributed Petascale Science Workflow Automation at DOE Facilities Automation Reproducibility Security Reusability Storage Metadata Analysis Visualization Advanced Photon Source
  • 15. Discovering Services
    • People create services (data or functions) …
    • which I discover (& decide whether to use) …
    • & compose to create a new function ...
    • & then publish as a new service.
    •  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
    •  I hope that this “someone else” can manage security, reliability, scalability, …
    ! ! “ Service-Oriented Science”, Science , 2005
  • 16. Discovering Services
    • Assume success
    • Syntax, semantics
    • Permissions
    • Reputation
    •  The ultimate arbiter?
    •  Types, ontologies
    •  Can I use it?
    •  Billions of services
    A B
  • 17. Discovery (1): Registries Globus
  • 18. Discovery (2): Standardized Vocabularies Core Services Grid Service Uses Terminology Described In Cancer Data Standards Repository Enterprise Vocabulary Services References Objects Defined in Service Metadata Publishes Subscribes to and Aggregates Queries Service Metadata Aggregated In Registers To Discovery Client API Index Service Globus
  • 19.  
  • 20. Discovery (3): Tagging & Social Networking GLOSS : Generalized Labels Over Scientific data Sources (Foster, Nestorov)
  • 21. Discovery (3): Tagging & Social Networking David de Roure, Carole Goble, et al.
  • 22. Composing Services
    • People create services (data or functions) …
    • which I discover (& decide whether to use) …
    • & compose to create a new function ...
    • & then publish as a new service.
    •  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
    •  I hope that this “someone else” can manage security, reliability, scalability, …
    ! ! “ Service-Oriented Science”, Science , 2005
  • 23. Composing Services: E.g., BPEL Workflow System Analytic service @ osu.edu Analytic service @ duke.edu caBiG: https://cabig.nci.nih.gov/; BPEL work: Ravi Madduri et al. link link link See also Kepler & Taverna Data Service @ uchicago.edu <BPEL Workflow Doc> <Workflow Inputs> <Workflow Results> BPEL Engine link Globus
  • 24. Composing Services: Taverna caGrid Scavenger with semantic/metadata- based caGrid service query A sample caGrid workflow Globus
  • 25. Composing Services Globus
  • 26. Demonstration: Composing Services Taverna + GT4 Taverna team Wei Tan Ravi Madduri
  • 27. Publishing Services
    • People create services (data or functions) …
    • which I discover (& decide whether to use) …
    • & compose to create a new function ...
    • & then publish as a new service.
    •  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
    •  I hope that this “someone else” can manage security, reliability, scalability, …
    ! ! “ Service-Oriented Science”, Science , 2005
  • 28. Publishing Services
    • Description  Syntax, semantics
    • State  Availability, load, …
    • Policies  Who, what, when, …
    • Hosting  Location, scalability, …
  • 29. Authorization: SAML & XACML VOMS Shibboleth LDAP PERMIS … GT4 Client GT4 Server PDP Attributes Authorization Decision PIP PIP PIP SAML XACML Globus
  • 30. Hosting Services
    • People create services (data or functions) …
    • which I discover (& decide whether to use) …
    • & compose to create a new function ...
    • & then publish as a new service.
    •  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!
    •  I hope that this “someone else” can manage security, reliability, scalability, …
    ! ! “ Service-Oriented Science”, Science , 2005
  • 31. The Two Dimensions of Service-Oriented Science
    • Decompose across network
    • Clients integrate dynamically
      • Select & compose services
      • Select “best of breed” providers
      • Publish result as new services
    • Decouple resource & service providers
    Function Resource Data Archives Analysis tools Discovery tools Users Fig: S. G. Djorgovski
  • 32. The geWorkbench/caGrid/TeraGrid Interface
  • 33. Putting It Together for the Example Scenario Location A Microarray, Protein, Image data Location B Microarray, Protein, Image data Location C Microarray, Protein, Image data Location C Image Analysis Location D Image Analysis caGrid Service Interfaces caGrid Environ-ment Registered Object Definitions Advertise-ment Log on, Grid credentials Query and Analysis Workflow Discovery Microarray & protein databases at other institutions
  • 34. Lessons Learned
    • A convenient higher-level abstraction
      • Suitable for a subset of scientific use cases
    • Infrastructure need to be sustainable
    • Integrates well with hospital/cancer center/experimental facility IT infrastructure
    • Workflows are attractive to users
    • Scalability and provenance are important
    • No vendor lock-in (if you are careful)
    • User experience remains ambiguous
      • Early adopters are enthusiastic (50+ services)
      • Cancer centers seek clear ROI
  • 35. Services for Science
    • A new approach to communicating
      • A (not-so new) approach to structuring systems
    • They’re real
      • Excellent infrastructure and tools (Globus, Introduce, gRAVI, Taverna, Swift, etc., etc.)
      • Substantial numbers of services out there
    • They’re challenging
      • Sociology: incentives, rewards
      • Infrastructure: hosting
      • Provenance: justifying “results”
      • Scaling: services, requests