• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
OGCE TeraGrid 2010 Science Gateway Tutorial Intro
 

OGCE TeraGrid 2010 Science Gateway Tutorial Intro

on

  • 1,413 views

OGCE TeraGrid 2010 Science Gateway Tutorial Intro, Pittsburgh, PA, August 2-5

OGCE TeraGrid 2010 Science Gateway Tutorial Intro, Pittsburgh, PA, August 2-5

Statistics

Views

Total Views
1,413
Views on SlideShare
959
Embed Views
454

Actions

Likes
0
Downloads
5
Comments
0

21 Embeds 454

http://collab-ogce.blogspot.com 374
http://collab-ogce.blogspot.in 26
http://collab-ogce.blogspot.fr 8
http://collab-ogce.blogspot.ca 7
http://collab-ogce.blogspot.it 6
http://collab-ogce.blogspot.co.uk 5
http://collab-ogce.blogspot.com.es 4
http://collab-ogce.blogspot.jp 4
http://collab-ogce.blogspot.com.br 3
http://collab-ogce.blogspot.de 3
http://collab-ogce.blogspot.co.at 2
http://collab-ogce.blogspot.be 2
http://translate.googleusercontent.com 2
http://webcache.googleusercontent.com 1
http://collab-ogce.blogspot.co.il 1
http://collab-ogce.blogspot.com.au 1
http://collab-ogce.blogspot.tw 1
http://collab-ogce.blogspot.com.ar 1
http://74.6.238.252 1
http://collab-ogce.blogspot.nl 1
http://collab-ogce.blogspot.kr 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Diversity of resources are critical to support diversity of user needs spanning research and education efforts August 2005 Charlie Catlett (catlett@mcs.anl.gov)
  • Cancer biomedical informatics grid

OGCE TeraGrid 2010 Science Gateway Tutorial Intro OGCE TeraGrid 2010 Science Gateway Tutorial Intro Presentation Transcript

  • CloudCom 2010
  • Gateway Computing Environments (GCE10)
  • Software for Science Gateways: Open Grid Computing Environments Marlon Pierce, Suresh Marru Pervasive Technology Institute Indiana University www.collab-ogce.org
  • Tutorial Agenda
    • 1:00-2:15
      • Introduction
      • Computational chemistry workflow example
      • Building the software
    • 2:15-2:30: Break
    • 2:30-5:00
      • Hands on workflow: OREChem
      • Hands on portal: Data mining
      • Demo: gadget container
  • http://www.collab-ogce.org/ogce/index.php/Tutorials Link to demonstration movies.
  • TeraGrid is one of the largest investments in shared CI from NSF’s Office of Cyberinfrastructure
  • TeraGrid resources today include:
    • Tightly Coupled Distributed Memory Systems, 2 systems in the top 10 at top500.org
      • Kraken (NICS): Cray XT5, 99,072 cores, 1.03 Pflop
      • Ranger (TACC): Sun Constellation, 62,976 cores, 579 Tflop, 123 TB RAM
    • Shared Memory Systems
      • Cobalt (NCSA): Altix, 8 Tflop, 3 TB shared memory
      • Pople (PSC): Altix, 5 Tflop, 1.5 TB shared memory
    • Clusters with Infiniband
      • Abe (NCSA): 90 Tflops
      • Lonestar (TACC): 61 Tflops
      • QueenBee (LONI): 51 Tflops
    • Condor Pool (Loosely Coupled)
      • Purdue- up to 22,000 cpus
    • Gateway hosting
      • Quarry (IU): virtual machine support
    • Visualization Resources
      • TeraDRE (Purdue): 48 node nVIDIA GPUs
      • Spur (TACC): 32 nVIDIA GPUs
    • Storage Resources
      • GPFS-WAN (SDSC)
      • Lustre-WAN (IU)
      • Various archival resources
    Source: Dan Katz, U Chicago
    • But change is constant - new systems:
    • Data Analysis and Vis systems
      • Longhorn (TACC): Dell/NVIDIA, CPU and GPU
      • Nautilus (NICS): SGI UltraViolet, 1024 cores, 4TB global shared memory
    • Data-Intensive Computing
      • Dash (SDSC): Intel Nehalem, 544 processors, 4TB flash memory
    • FutureGrid
      • Experimental computing grid and cloud test-bed to tackle research challenges in computer science
    • Keeneland
      • Experimental, high-performance computing system with NVIDIA Tesla accelerators
  • What Is a Science Gateway?
    • Web and desktop user interfaces and user-centric Web services for accessing Grid and Cloud resources.
      • Clusters, supercomputers, mass storage
      • Applications, databases
      • Workflows
    • Example Science Gateways from the NSF TeraGrid
      • GridChem: computational chemistry
      • UltraScan: biophysics computational analysis
      • LEAD: Atmospheric science
      • BioDrugScreen: drug docking, scoring, and discovery.
    • Many others: see https://www.teragrid.org/web/science-gateways/gateway_list
    • This tutorial is about software that powers gateways.
  • When is a gateway appropriate?
    • Provide access to community applications
      • WRF, Gaussian, CHARMM, Amber, BLAST, CCSM, UltraScan
      • Create multi-scale workflows
    • Provide access to community data sets
      • National Virtual Observatory
      • Earth System Grid
      • Some groups have invested significant efforts here
        • caBIG, extensive discussions to develop common terminology and formats
        • BIRN, extensive data sharing agreements
    • Difficult to access data/advanced workflows
      • Sensor/radar input
        • LEAD, GEON
  • 3 steps to connect a gateway to TeraGrid
    • Request an allocation
      • Only a 1 paragraph abstract required for up to 200k CPU hours
    • Register your gateway
      • Visibility on public TeraGrid
      • page
    • Request a community account
      • Run jobs for others via your portal
    • Staff support is available!
    • www.teragrid.org/gateways
    SciDAC, Chattanooga, TN, July 16, 2010
  • OVP/ RST/ MIG OGCE Re-engineer, Generalize, Build, Test and Release LEAD OGCE Gateway Tool Adaption & Reuse GridChem TeraGrid User Portal OGCE Team GridChem Ultrascan BioVLab ODI Bio Drug Screen EST Pipeline Future Grid GFac , XBaya , XRegistry, FTR Eventing System LEAD Resource Discovery Service GPIR, File Browser Gadget Container , GTLab, Javascript Cog, XRegistry Interface, Experiment Builder, Axis2 Gfac , Axis2 Eventing System, Resource Prediction Service, Swarm Experiment Builder, XRegistry Interface Xbaya, GC Middleware GFac , Eventing System XBaya, GFac Workflow Suite, Gadget Container Swarm-> GFac Swarm-> GFac GFac, Xbaya , …
  • Software Description OGCE Gadget Container Google Gadget/Open Social compatible software for building Web-based user interfaces. XBaya A visual user interface for composing, launching and monitoring workflows GFAC An application factory service for wrapping command-line tools as Web services XRegistry; Registry Gadget A service and workflow registry and its user interface Experiment Builder User interface for creating online experiments with registered workflows
  • Science Gateways Layer Cake Color Coding Dependent resource provider components Complimentary Gateway Components OGCE Gateway Components Compute Resources Resource Middleware Cloud Interfaces Grid Middleware SSH & Resource Managers Computational Clouds Computational Grids Gateway Services User Interfaces Web/Gadget Container Web Enabled Desktop Applications User Management Auditing & Reporting Fault Tolerance Application Abstractions Workflow System Information Services Application Monitoring Registry Security Provenance & Metadata Management Local Resources Web/Gadget Interfaces Gateway Abstraction Interfaces
  • GFac Current & Future Features Color Coding Planned/Requested Features Existing Features Input Handlers Scheduling Interface Auditing Monitoring Interface Data Management Abstraction Job Management Abstraction Fault Tolerance Output Handlers Registry Interface Checkpoint Support Globus Campus Resources Unicore Condor Amazon Eucalyptus
  • OGCE Layered Workflow Architecture: Derived from LEAD Workflow System Workflow Execution & Control Engines Apache ODE Workflow Specification Workflow Interfaces (Design & Definition) Python BPEL 2.0 BPEL 1.0 Java Code Pegasus DAG Scufl XBaya GUI (Composition, Deploying, Steering & Monitoring) Gadget Interface for Input Binding Condor DAGMan Taverna Dynamic Enactor Jython Interpreter GBPEL Flex/Web Composition
  • Putting It All Together
  • Software Strategy
    • Focus on gadget container and tools for running science applications on grids and clouds.
    • Provide a tool set that can be used in whole or in part.
      • If you just want GFac, then you can use it without buying an entire framework.
    • Outsource security, information services, data and metadata, etc to other providers.
      • MyProxy, TG IIS, Globus, Condor, XMC Cat, iRods, etc.
  • More Information
    • This is downloadable, packaged software.
      • Apache Maven build system provides everything you need to to build the gadget container, gadgets, workflow composer, and backing services.
      • Get code by anonymous SVN checkout.
    • Email: [email_address] , [email_address] , [email_address]
    • OGCE Web Site: www.collab-ogce.org
    • Blog/News Feed: http://collab-ogce.blogspot.com/
  • Acknowledgements and People
    • Funding by TeraGrid GIG, RP and by OCI SDCI
    • IU: Marlon Pierce, Suresh Marru, Raminder Singh, Archit Kulshrestha, Zhenhua Guo
    • TACC: Maytal Dahan, Rion Dooley
    • SDSC: Nancy Wilkins-Diehr, Jeff Sale
    • SDSU: Mary Thomas
  • Demos Next
  • The OGCE Application Registry gadget allows users to interactively register hosts and applications that are dynamically wrapped as Web services.
  • The OGCE Gadget Container allows you to build portals out of public and private Google Open Social gadgets. Supports HTTPS. Downloadable, packaged software.
  • The OGCE Experiment Builder gadget allows users to create projects and experiments out of previously composed workflows.
  • OGCE Tools for Science Workflows The XBaya workflow composer allows you to build scientific workflows from services running across the TeraGrid. This is part of our workflow suite.
  • What Is a Science Gateway?
    • Web and desktop user interfaces and user-centric Web services for accessing Grid and Cloud resources.
      • Clusters, supercomputers, mass storage
      • Applications, databases
      • Workflows
    • Example Science Gateways from the NSF TeraGrid
      • GridChem: computational chemistry
      • UltraScan: biophysics computational analysis
      • LEAD: Atmospheric science
      • BioDrugScreen: drug docking, scoring, and discovery.
    • Many others: see https://www.teragrid.org/web/science-gateways/gateway_list
    • This demo is about software that powers gateways.
  • Google Gadget-Based Science Gateways LEAD PolarGrid