SlideShare a Scribd company logo
1 of 49
Science Gateway Advanced Support Activities in PTI Marlon Pierce Indiana University
OGCE Gateway Tool Adaption & Reuse LEAD LEAD Experiment Builder, XRegistry Interface GFac, XBaya, XRegistry, FTR Eventing System GridChem XBaya GridChem Ultrascan OGCE Re-engineer, Generalize, Build, Test and Release GFac, Eventing System Resource Discovery Service BioVLab OVP/ RST/  MIG  TeraGrid User Portal XBaya, GFac ODI GPIR, File Browser Workflow Suite, Gadget Container OGCE Team Bio Drug Screen Swarm->GFac Gadget Container, GTLab, Javascript Cog, XRegistry Interface, Experiment Builder, Axis2Gfac, Axis2 Eventing System, Resource Prediction Service, Swarm EST Pipeline Swarm->GFac Future Grid GFac, Xbaya, … 3 3
Gateway Hosting Service ,[object Object]
 This has been a valuable resource for our group.
 We should look for ways to expand its usage, such as supporting data collectionsCourtesy: Mike Lowe, Dave Hancock.
BioDrugScreen Portal Support: Josh Rosen and ArchitKulshrestha Collaboration: SamyMeroueh, IUPUI
BioDrugScreen A computational drug discovery resource Contains millions of pre-docked and pre-scored complexes between thousands of targets from the human proteome and thousands of drug-like small molecules Allows drug researchers to develop their own scoring functions of calculating how well a drug will interact with a protein.
Small drug-like molecules from NCI diversity set are docked into 205 proteasome protein target.  Orange area is an identified target area of the protein. Visualization uses JMol. Docking done with Amber on the TeraGrid.  Proteins are obtained from the PDB.  Samy maintains his own database of protein structures and small molecules.
Support Provided Proteins that have not had their docking and scores calculated need to have these run on the Teragrid.  An Web interface needed to be created so users can submit their own jobs. We developed the interface between the site’s submissions and the Teragrid using the Swarm Service. Also prototyping GFac support Using Flash and javascript, we developed an improved data presentation for the ranking page
UltraScan Gateway Staff: Raminder Singh Collaborators: BorriesDemeler and Emre Brookes, UTHSCSA
UltraScan Science Gateway A biophysics gateway for investigating properties and structure-function relationships of biological macromolecules, nanoparticles, polymers and colloids that are implicated in many diseases, including cancer. High-resolution analysis and modeling of hydrodynamic data from an analytical ultracentrifuge. TeraGrid as a backup spill over resources but still is one of the heaviest user consuming 1.75 million SU’s in 6 months.
UltraScan Advanced Support Porting to new architectures and parallel performance enhancements. New workflow implementations, new grid computing and grid middleware support: Reliability problems with WSGram Missing job status Only supports Gram4, needs porting to other middleware Issues with data movement. Need Fault tolerance at all levels. Users decide resources manually, need automated scheduling. Current Architecture
UltraScan OGCE Integration Enhance the perl job submission daemon and monitoring with  OGCE GFacservice. Implement and iteratively enhance fault tolerance. Port to community account usage withGridshibauditing support. Support UNICORE to run jobs on other European and Australian resources.
GridChem Support: Suresh Marru, Raminder Singh Collaborators: SudhakarPamidighantam, NCSA
GridChem Science Gateway A chemistry/material Science Gateway for running computational chemistry codes, workflows, and parameter sweeps. Integrates molecular science applications and tools for community use.  400+ users heavily using TeraGrid. One of the consistentTop 5 TeraGrid Gateway users. Supports all popular Chemistry applications including Gaussian, GAMESS, NWChem, QMCPack, Amber and MolPro, CHARMM
GridChem Advanced Support GridChem supports single application executions Advanced support request for supporting workflows Improved Fault Tolerance
GridChem OGCE Integration OGCE workflow tools wrapped Gaussian & CHARMM chemistry applications Coupled Butane workflow using Gaussian & CHARMM Integration 100 member Gaussian parametric sweeps Integration with Pegasus workflow tools Ye Fan, Master’s student
GridChem Using OGCE Tools Initial Structure Optimized Structure GridChem using OGCE Workflow Tools to construct and execute CHARMM and Gaussian Molecular chemistry Models
Future Grid User Portal Support: SiddMaini, ArchitKulshrestha
Future Grid User Portal Our strategy is to build all components as Google Gadgets that interact with REST Services Can live in iGoogle as well as containers like Drupal. Take advantage of OpenID and OAuth Initial target gadgets: Knowledge Base, Amazon EC2 Clients, Inca Clients Future Work: services, gadgets, and workflows for managing machine images with Xcat.
Future Grid KnowledgeBase (FGKB) Task:  Develop FGKB Web App Search KB Documents Technology used: Adobe Flex / PHP / KB REST API
Current Status: Basic Search and Retrieval Live URL: http://tinyurl.com/ykaa9gr
EC2 Client User Interface Link: http://tinyurl.com/ylkohj7 ,[object Object]
Launch them
Terminate themNext Step: Add more EC2 features, integrate with FutureGrid user database
Portal Embedded Gadgets
FutureGrid Machine Image Services ,[object Object]
On both real hardware and Virtual machines.
We are working to capture common XCat tasks as scripts
These scripts can be wrapped as secure services using OGCE’sGFac.
Several tasks can be linked together as workflows visually composed with OGCE’sXBaya.
You can still use Pegasus/Condor as the workflow engine ,[object Object]
Science Gateways Layer Cake User Interfaces Gateway Abstraction Interfaces Web Enabled Desktop Applications Web/Gadget Container  Web/Gadget Interfaces Information Services Application Abstractions Application Monitoring User Management Fault Tolerance  Gateway Services Provenance & Metadata Management Registry Workflow System Security Auditing & Reporting Resource Middleware Cloud Interfaces SSH & Resource Managers Grid Middleware Compute Resources Local Resources Computational Grids Computational Clouds Color Coding OGCE Gateway Components Complimentary Gateway Components Dependent resource provider components
Google Gadget-Based Science Gateways PolarGrid MyOSG LEAD
GFac Current & Future Features Apache  Axis2  Globus Registry Interface Scheduling Interface Input Handlers Monitoring Interface Campus Resources Data Management Abstraction Fault Tolerance  Output Handlers Amazon Eucalyptus Job Management Abstraction Auditing Checkpoint Support Unicore Condor Color Coding Existing Features Planned/Requested Features
OGCE  Layered Workflow Architecture:Derived from LEAD Workflow System Workflow Interfaces (Design & Definition)  XBaya GUI (Composition, Deploying, Steering & Monitoring)  Flex/Web Composition Gadget Interface for Input Binding Python BPEL 2.0 Workflow Specification Scufl BPEL 1.0 Java Code  Pegasus DAG Workflow Execution & Control  Engines Apache ODE Condor DAGMan Dynamic Enactor Jython Interpreter GBPEL Taverna
Putting It All Together
Software Strategy Focus on gadget container and tools for running science applications on grids and clouds. Provide a tool set that can be used in whole or in part. If you just want GFac, then you can use it without buying an entire framework. Outsource security, information services, data and metadata, advanced job handling, etc to other providers. MyProxy, TG IIS, Globus, Condor, XMC Cat, iRods, etc.
Packaging, Building, and Testing All builds are designed to be self contained  Use Apache Maven 2.x Download includes everything you need. Portal, Axis Services, and Xregistry all build nightly on the NMI Build and Test facility at UW. Several Linux platforms, Mac PPC, and Mac X86 Java 1.5 Apache JMetertest suite for the portal.   Run against your installation Automated tests nightly
Next Steps Apache Incubator Project for XBaya, GFac and supporting workflow tools WIYN ODI instrument pipeline and gateway Robert Henschel and Scott Michael are leading overall effort. Suresh and Raminder are working 50% time through early April on technical evaluation of integrating NHPPS software with OGCE New Collaboration: Craig Mattocks, UNC will build a storm surge forecasting gateway Broadly similar to LEAD and SCOOP Archit will be the point of contact Local gateway tutorial in early April Sun Kim’s group, UNC Group, CGB Others welcome Gadget Container additional applications  OGCE grid gadgets packaged release, SimpleGrid
More Information, Acknowledgements Website: www.collab-ogce.org Blog/RSS Feed: collab-ogce.blogspot.com Email: mpierce@cs.indiana.edu, smarru@cs.indiana.edu Geoffrey Fox, Craig Stewart, and Dennis Gannon have spent years laying the foundation for this work
Backup Slides
EST Assembly Pipeline Support: ArchitKulshrestha, Chin Hua Kong Collaborator: Qunfeng Dong, UNT
[object Object]
Computation is outsourced to Grids (TeraGrid) and Clouds (Amazon)
Not provided by in-house clusters.
This is an open service, open architecture approach.
These codes have very different scheduling requirementshttp://swarm.cgb.indiana.edu

More Related Content

What's hot

Scalable Monitoring Using Prometheus with Apache Spark Clusters with Diane F...
 Scalable Monitoring Using Prometheus with Apache Spark Clusters with Diane F... Scalable Monitoring Using Prometheus with Apache Spark Clusters with Diane F...
Scalable Monitoring Using Prometheus with Apache Spark Clusters with Diane F...Databricks
 
Computing Outside The Box September 2009
Computing Outside The Box September 2009Computing Outside The Box September 2009
Computing Outside The Box September 2009Ian Foster
 
Stream Processing with CompletableFuture and Flow in Java 9
Stream Processing with CompletableFuture and Flow in Java 9Stream Processing with CompletableFuture and Flow in Java 9
Stream Processing with CompletableFuture and Flow in Java 9Trayan Iliev
 
Microservices with Spring 5 Webflux - jProfessionals
Microservices  with Spring 5 Webflux - jProfessionalsMicroservices  with Spring 5 Webflux - jProfessionals
Microservices with Spring 5 Webflux - jProfessionalsTrayan Iliev
 
Oracle cloud data interface
Oracle cloud data interfaceOracle cloud data interface
Oracle cloud data interfaceOracle Korea
 
Galaxy
GalaxyGalaxy
Galaxybosc
 
Spring boot microservice metrics monitoring
Spring boot   microservice metrics monitoringSpring boot   microservice metrics monitoring
Spring boot microservice metrics monitoringOracle Korea
 
Dataservices - Processing Big Data The Microservice Way
Dataservices - Processing Big Data The Microservice WayDataservices - Processing Big Data The Microservice Way
Dataservices - Processing Big Data The Microservice WayJosef Adersberger
 
The Materials Project Ecosystem - A Complete Software and Data Platform for M...
The Materials Project Ecosystem - A Complete Software and Data Platform for M...The Materials Project Ecosystem - A Complete Software and Data Platform for M...
The Materials Project Ecosystem - A Complete Software and Data Platform for M...University of California, San Diego
 
Real-Time Log Analysis with Apache Mesos, Kafka and Cassandra
Real-Time Log Analysis with Apache Mesos, Kafka and CassandraReal-Time Log Analysis with Apache Mesos, Kafka and Cassandra
Real-Time Log Analysis with Apache Mesos, Kafka and CassandraJoe Stein
 
Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...
Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...
Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...Spark Summit
 
Spring 5 Webflux - Advances in Java 2018
Spring 5 Webflux - Advances in Java 2018Spring 5 Webflux - Advances in Java 2018
Spring 5 Webflux - Advances in Java 2018Trayan Iliev
 
Computing Outside The Box June 2009
Computing Outside The Box June 2009Computing Outside The Box June 2009
Computing Outside The Box June 2009Ian Foster
 
Real time machine learning visualization with spark -- Hadoop Summit 2016
Real time machine learning visualization with spark -- Hadoop Summit 2016Real time machine learning visualization with spark -- Hadoop Summit 2016
Real time machine learning visualization with spark -- Hadoop Summit 2016Chester Chen
 
Reactive Microservices with Spring 5: WebFlux
Reactive Microservices with Spring 5: WebFlux Reactive Microservices with Spring 5: WebFlux
Reactive Microservices with Spring 5: WebFlux Trayan Iliev
 
Apache Kafka Streams + Machine Learning / Deep Learning
Apache Kafka Streams + Machine Learning / Deep LearningApache Kafka Streams + Machine Learning / Deep Learning
Apache Kafka Streams + Machine Learning / Deep LearningKai Wähner
 
C2MON - A highly scalable monitoring platform for Big Data scenarios @CERN by...
C2MON - A highly scalable monitoring platform for Big Data scenarios @CERN by...C2MON - A highly scalable monitoring platform for Big Data scenarios @CERN by...
C2MON - A highly scalable monitoring platform for Big Data scenarios @CERN by...J On The Beach
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Rapid Web API development with Kotlin and Ktor
Rapid Web API development with Kotlin and KtorRapid Web API development with Kotlin and Ktor
Rapid Web API development with Kotlin and KtorTrayan Iliev
 

What's hot (20)

Scalable Monitoring Using Prometheus with Apache Spark Clusters with Diane F...
 Scalable Monitoring Using Prometheus with Apache Spark Clusters with Diane F... Scalable Monitoring Using Prometheus with Apache Spark Clusters with Diane F...
Scalable Monitoring Using Prometheus with Apache Spark Clusters with Diane F...
 
Computing Outside The Box September 2009
Computing Outside The Box September 2009Computing Outside The Box September 2009
Computing Outside The Box September 2009
 
Stream Processing with CompletableFuture and Flow in Java 9
Stream Processing with CompletableFuture and Flow in Java 9Stream Processing with CompletableFuture and Flow in Java 9
Stream Processing with CompletableFuture and Flow in Java 9
 
The Materials API
The Materials APIThe Materials API
The Materials API
 
Microservices with Spring 5 Webflux - jProfessionals
Microservices  with Spring 5 Webflux - jProfessionalsMicroservices  with Spring 5 Webflux - jProfessionals
Microservices with Spring 5 Webflux - jProfessionals
 
Oracle cloud data interface
Oracle cloud data interfaceOracle cloud data interface
Oracle cloud data interface
 
Galaxy
GalaxyGalaxy
Galaxy
 
Spring boot microservice metrics monitoring
Spring boot   microservice metrics monitoringSpring boot   microservice metrics monitoring
Spring boot microservice metrics monitoring
 
Dataservices - Processing Big Data The Microservice Way
Dataservices - Processing Big Data The Microservice WayDataservices - Processing Big Data The Microservice Way
Dataservices - Processing Big Data The Microservice Way
 
The Materials Project Ecosystem - A Complete Software and Data Platform for M...
The Materials Project Ecosystem - A Complete Software and Data Platform for M...The Materials Project Ecosystem - A Complete Software and Data Platform for M...
The Materials Project Ecosystem - A Complete Software and Data Platform for M...
 
Real-Time Log Analysis with Apache Mesos, Kafka and Cassandra
Real-Time Log Analysis with Apache Mesos, Kafka and CassandraReal-Time Log Analysis with Apache Mesos, Kafka and Cassandra
Real-Time Log Analysis with Apache Mesos, Kafka and Cassandra
 
Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...
Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...
Accelerating Apache Spark-based Analytics on Intel Architecture-(Michael Gree...
 
Spring 5 Webflux - Advances in Java 2018
Spring 5 Webflux - Advances in Java 2018Spring 5 Webflux - Advances in Java 2018
Spring 5 Webflux - Advances in Java 2018
 
Computing Outside The Box June 2009
Computing Outside The Box June 2009Computing Outside The Box June 2009
Computing Outside The Box June 2009
 
Real time machine learning visualization with spark -- Hadoop Summit 2016
Real time machine learning visualization with spark -- Hadoop Summit 2016Real time machine learning visualization with spark -- Hadoop Summit 2016
Real time machine learning visualization with spark -- Hadoop Summit 2016
 
Reactive Microservices with Spring 5: WebFlux
Reactive Microservices with Spring 5: WebFlux Reactive Microservices with Spring 5: WebFlux
Reactive Microservices with Spring 5: WebFlux
 
Apache Kafka Streams + Machine Learning / Deep Learning
Apache Kafka Streams + Machine Learning / Deep LearningApache Kafka Streams + Machine Learning / Deep Learning
Apache Kafka Streams + Machine Learning / Deep Learning
 
C2MON - A highly scalable monitoring platform for Big Data scenarios @CERN by...
C2MON - A highly scalable monitoring platform for Big Data scenarios @CERN by...C2MON - A highly scalable monitoring platform for Big Data scenarios @CERN by...
C2MON - A highly scalable monitoring platform for Big Data scenarios @CERN by...
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream Processing
 
Rapid Web API development with Kotlin and Ktor
Rapid Web API development with Kotlin and KtorRapid Web API development with Kotlin and Ktor
Rapid Web API development with Kotlin and Ktor
 

Similar to Indiana University's Advanced Science Gateway Support

OGCE RT Rroject Review
OGCE RT Rroject ReviewOGCE RT Rroject Review
OGCE RT Rroject Reviewmarpierc
 
OGCE Review for Indiana University Research Technologies
OGCE Review for Indiana University Research TechnologiesOGCE Review for Indiana University Research Technologies
OGCE Review for Indiana University Research Technologiesmarpierc
 
OGCE SciDAC2010 Tutorial
OGCE SciDAC2010 TutorialOGCE SciDAC2010 Tutorial
OGCE SciDAC2010 Tutorialmarpierc
 
Building Science Gateways with Gadgets and OpenSocial
Building Science Gateways with Gadgets and OpenSocialBuilding Science Gateways with Gadgets and OpenSocial
Building Science Gateways with Gadgets and OpenSocialmarpierc
 
OGCE Overview for SciDAC 2009
OGCE Overview for SciDAC 2009OGCE Overview for SciDAC 2009
OGCE Overview for SciDAC 2009marpierc
 
Ogce Workflow Suite
Ogce Workflow SuiteOgce Workflow Suite
Ogce Workflow Suitesmarru
 
XSEDE14 SciGaP-Apache Airavata Tutorial
XSEDE14 SciGaP-Apache Airavata TutorialXSEDE14 SciGaP-Apache Airavata Tutorial
XSEDE14 SciGaP-Apache Airavata Tutorialmarpierc
 
OGCE TG09 Tech Track Presentation
OGCE TG09 Tech Track PresentationOGCE TG09 Tech Track Presentation
OGCE TG09 Tech Track Presentationmarpierc
 
grid mining
grid mininggrid mining
grid miningARNOLD
 
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...NETWAYS
 
Computing Outside The Box
Computing Outside The BoxComputing Outside The Box
Computing Outside The BoxIan Foster
 
Grid Projects In The US July 2008
Grid Projects In The US July 2008Grid Projects In The US July 2008
Grid Projects In The US July 2008Ian Foster
 
GaurabDey_UFL_CV_fulltime (1)
GaurabDey_UFL_CV_fulltime (1)GaurabDey_UFL_CV_fulltime (1)
GaurabDey_UFL_CV_fulltime (1)Gaurab Dey
 
FAIR Computational Workflows
FAIR Computational WorkflowsFAIR Computational Workflows
FAIR Computational WorkflowsCarole Goble
 
Webinar september 2013
Webinar september 2013Webinar september 2013
Webinar september 2013Marc Gille
 
MPLS/SDN 2013 Intercloud Standardization and Testbeds - Sill
MPLS/SDN 2013 Intercloud Standardization and Testbeds - SillMPLS/SDN 2013 Intercloud Standardization and Testbeds - Sill
MPLS/SDN 2013 Intercloud Standardization and Testbeds - SillAlan Sill
 
Grafana overview deck - Tech - 2023 May v1.pdf
Grafana overview deck  - Tech - 2023 May v1.pdfGrafana overview deck  - Tech - 2023 May v1.pdf
Grafana overview deck - Tech - 2023 May v1.pdfBillySin5
 

Similar to Indiana University's Advanced Science Gateway Support (20)

OGCE RT Rroject Review
OGCE RT Rroject ReviewOGCE RT Rroject Review
OGCE RT Rroject Review
 
OGCE Review for Indiana University Research Technologies
OGCE Review for Indiana University Research TechnologiesOGCE Review for Indiana University Research Technologies
OGCE Review for Indiana University Research Technologies
 
OGCE SciDAC2010 Tutorial
OGCE SciDAC2010 TutorialOGCE SciDAC2010 Tutorial
OGCE SciDAC2010 Tutorial
 
OGCE SC10
OGCE SC10OGCE SC10
OGCE SC10
 
Building Science Gateways with Gadgets and OpenSocial
Building Science Gateways with Gadgets and OpenSocialBuilding Science Gateways with Gadgets and OpenSocial
Building Science Gateways with Gadgets and OpenSocial
 
OGCE Overview for SciDAC 2009
OGCE Overview for SciDAC 2009OGCE Overview for SciDAC 2009
OGCE Overview for SciDAC 2009
 
Ogce Workflow Suite
Ogce Workflow SuiteOgce Workflow Suite
Ogce Workflow Suite
 
XSEDE14 SciGaP-Apache Airavata Tutorial
XSEDE14 SciGaP-Apache Airavata TutorialXSEDE14 SciGaP-Apache Airavata Tutorial
XSEDE14 SciGaP-Apache Airavata Tutorial
 
OGCE TG09 Tech Track Presentation
OGCE TG09 Tech Track PresentationOGCE TG09 Tech Track Presentation
OGCE TG09 Tech Track Presentation
 
grid mining
grid mininggrid mining
grid mining
 
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
OSMC 2022 | Unifying Observability Weaving Prometheus, Jaeger, and Open Sourc...
 
Grid computing
Grid computingGrid computing
Grid computing
 
Computing Outside The Box
Computing Outside The BoxComputing Outside The Box
Computing Outside The Box
 
mitra_resume-2
mitra_resume-2mitra_resume-2
mitra_resume-2
 
Grid Projects In The US July 2008
Grid Projects In The US July 2008Grid Projects In The US July 2008
Grid Projects In The US July 2008
 
GaurabDey_UFL_CV_fulltime (1)
GaurabDey_UFL_CV_fulltime (1)GaurabDey_UFL_CV_fulltime (1)
GaurabDey_UFL_CV_fulltime (1)
 
FAIR Computational Workflows
FAIR Computational WorkflowsFAIR Computational Workflows
FAIR Computational Workflows
 
Webinar september 2013
Webinar september 2013Webinar september 2013
Webinar september 2013
 
MPLS/SDN 2013 Intercloud Standardization and Testbeds - Sill
MPLS/SDN 2013 Intercloud Standardization and Testbeds - SillMPLS/SDN 2013 Intercloud Standardization and Testbeds - Sill
MPLS/SDN 2013 Intercloud Standardization and Testbeds - Sill
 
Grafana overview deck - Tech - 2023 May v1.pdf
Grafana overview deck  - Tech - 2023 May v1.pdfGrafana overview deck  - Tech - 2023 May v1.pdf
Grafana overview deck - Tech - 2023 May v1.pdf
 

More from marpierc

IWSG2014: Developing Science Gateways Using Apache Airavata
IWSG2014: Developing Science Gateways Using Apache AiravataIWSG2014: Developing Science Gateways Using Apache Airavata
IWSG2014: Developing Science Gateways Using Apache Airavatamarpierc
 
ACES QuakeSim 2011
ACES QuakeSim 2011ACES QuakeSim 2011
ACES QuakeSim 2011marpierc
 
GCE11 Apache Rave Presentation
GCE11 Apache Rave PresentationGCE11 Apache Rave Presentation
GCE11 Apache Rave Presentationmarpierc
 
TG11 ORPS Poster
TG11 ORPS PosterTG11 ORPS Poster
TG11 ORPS Postermarpierc
 
SC11 Science Gateway Group Overview
SC11 Science Gateway Group OverviewSC11 Science Gateway Group Overview
SC11 Science Gateway Group Overviewmarpierc
 
Experiences with the Apache Software Foundation
Experiences with the Apache Software Foundation Experiences with the Apache Software Foundation
Experiences with the Apache Software Foundation marpierc
 
PNNL April 2011 ogce
PNNL April 2011 ogcePNNL April 2011 ogce
PNNL April 2011 ogcemarpierc
 
Ogce about-sc10
Ogce about-sc10Ogce about-sc10
Ogce about-sc10marpierc
 
GTLAB Installation Tutorial for SciDAC 2009
GTLAB Installation Tutorial for SciDAC 2009GTLAB Installation Tutorial for SciDAC 2009
GTLAB Installation Tutorial for SciDAC 2009marpierc
 
GTLAB Overview
GTLAB OverviewGTLAB Overview
GTLAB Overviewmarpierc
 

More from marpierc (10)

IWSG2014: Developing Science Gateways Using Apache Airavata
IWSG2014: Developing Science Gateways Using Apache AiravataIWSG2014: Developing Science Gateways Using Apache Airavata
IWSG2014: Developing Science Gateways Using Apache Airavata
 
ACES QuakeSim 2011
ACES QuakeSim 2011ACES QuakeSim 2011
ACES QuakeSim 2011
 
GCE11 Apache Rave Presentation
GCE11 Apache Rave PresentationGCE11 Apache Rave Presentation
GCE11 Apache Rave Presentation
 
TG11 ORPS Poster
TG11 ORPS PosterTG11 ORPS Poster
TG11 ORPS Poster
 
SC11 Science Gateway Group Overview
SC11 Science Gateway Group OverviewSC11 Science Gateway Group Overview
SC11 Science Gateway Group Overview
 
Experiences with the Apache Software Foundation
Experiences with the Apache Software Foundation Experiences with the Apache Software Foundation
Experiences with the Apache Software Foundation
 
PNNL April 2011 ogce
PNNL April 2011 ogcePNNL April 2011 ogce
PNNL April 2011 ogce
 
Ogce about-sc10
Ogce about-sc10Ogce about-sc10
Ogce about-sc10
 
GTLAB Installation Tutorial for SciDAC 2009
GTLAB Installation Tutorial for SciDAC 2009GTLAB Installation Tutorial for SciDAC 2009
GTLAB Installation Tutorial for SciDAC 2009
 
GTLAB Overview
GTLAB OverviewGTLAB Overview
GTLAB Overview
 

Recently uploaded

Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraDeakin University
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptxLBM Solutions
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsPrecisely
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 

Recently uploaded (20)

Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning era
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
Key Features Of Token Development (1).pptx
Key  Features Of Token  Development (1).pptxKey  Features Of Token  Development (1).pptx
Key Features Of Token Development (1).pptx
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
The transition to renewables in India.pdf
The transition to renewables in India.pdfThe transition to renewables in India.pdf
The transition to renewables in India.pdf
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power Systems
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 

Indiana University's Advanced Science Gateway Support

  • 1. Science Gateway Advanced Support Activities in PTI Marlon Pierce Indiana University
  • 2.
  • 3. OGCE Gateway Tool Adaption & Reuse LEAD LEAD Experiment Builder, XRegistry Interface GFac, XBaya, XRegistry, FTR Eventing System GridChem XBaya GridChem Ultrascan OGCE Re-engineer, Generalize, Build, Test and Release GFac, Eventing System Resource Discovery Service BioVLab OVP/ RST/ MIG TeraGrid User Portal XBaya, GFac ODI GPIR, File Browser Workflow Suite, Gadget Container OGCE Team Bio Drug Screen Swarm->GFac Gadget Container, GTLab, Javascript Cog, XRegistry Interface, Experiment Builder, Axis2Gfac, Axis2 Eventing System, Resource Prediction Service, Swarm EST Pipeline Swarm->GFac Future Grid GFac, Xbaya, … 3 3
  • 4.
  • 5. This has been a valuable resource for our group.
  • 6. We should look for ways to expand its usage, such as supporting data collectionsCourtesy: Mike Lowe, Dave Hancock.
  • 7. BioDrugScreen Portal Support: Josh Rosen and ArchitKulshrestha Collaboration: SamyMeroueh, IUPUI
  • 8. BioDrugScreen A computational drug discovery resource Contains millions of pre-docked and pre-scored complexes between thousands of targets from the human proteome and thousands of drug-like small molecules Allows drug researchers to develop their own scoring functions of calculating how well a drug will interact with a protein.
  • 9. Small drug-like molecules from NCI diversity set are docked into 205 proteasome protein target. Orange area is an identified target area of the protein. Visualization uses JMol. Docking done with Amber on the TeraGrid. Proteins are obtained from the PDB. Samy maintains his own database of protein structures and small molecules.
  • 10. Support Provided Proteins that have not had their docking and scores calculated need to have these run on the Teragrid. An Web interface needed to be created so users can submit their own jobs. We developed the interface between the site’s submissions and the Teragrid using the Swarm Service. Also prototyping GFac support Using Flash and javascript, we developed an improved data presentation for the ranking page
  • 11. UltraScan Gateway Staff: Raminder Singh Collaborators: BorriesDemeler and Emre Brookes, UTHSCSA
  • 12. UltraScan Science Gateway A biophysics gateway for investigating properties and structure-function relationships of biological macromolecules, nanoparticles, polymers and colloids that are implicated in many diseases, including cancer. High-resolution analysis and modeling of hydrodynamic data from an analytical ultracentrifuge. TeraGrid as a backup spill over resources but still is one of the heaviest user consuming 1.75 million SU’s in 6 months.
  • 13. UltraScan Advanced Support Porting to new architectures and parallel performance enhancements. New workflow implementations, new grid computing and grid middleware support: Reliability problems with WSGram Missing job status Only supports Gram4, needs porting to other middleware Issues with data movement. Need Fault tolerance at all levels. Users decide resources manually, need automated scheduling. Current Architecture
  • 14. UltraScan OGCE Integration Enhance the perl job submission daemon and monitoring with OGCE GFacservice. Implement and iteratively enhance fault tolerance. Port to community account usage withGridshibauditing support. Support UNICORE to run jobs on other European and Australian resources.
  • 15. GridChem Support: Suresh Marru, Raminder Singh Collaborators: SudhakarPamidighantam, NCSA
  • 16. GridChem Science Gateway A chemistry/material Science Gateway for running computational chemistry codes, workflows, and parameter sweeps. Integrates molecular science applications and tools for community use. 400+ users heavily using TeraGrid. One of the consistentTop 5 TeraGrid Gateway users. Supports all popular Chemistry applications including Gaussian, GAMESS, NWChem, QMCPack, Amber and MolPro, CHARMM
  • 17. GridChem Advanced Support GridChem supports single application executions Advanced support request for supporting workflows Improved Fault Tolerance
  • 18. GridChem OGCE Integration OGCE workflow tools wrapped Gaussian & CHARMM chemistry applications Coupled Butane workflow using Gaussian & CHARMM Integration 100 member Gaussian parametric sweeps Integration with Pegasus workflow tools Ye Fan, Master’s student
  • 19. GridChem Using OGCE Tools Initial Structure Optimized Structure GridChem using OGCE Workflow Tools to construct and execute CHARMM and Gaussian Molecular chemistry Models
  • 20. Future Grid User Portal Support: SiddMaini, ArchitKulshrestha
  • 21. Future Grid User Portal Our strategy is to build all components as Google Gadgets that interact with REST Services Can live in iGoogle as well as containers like Drupal. Take advantage of OpenID and OAuth Initial target gadgets: Knowledge Base, Amazon EC2 Clients, Inca Clients Future Work: services, gadgets, and workflows for managing machine images with Xcat.
  • 22. Future Grid KnowledgeBase (FGKB) Task: Develop FGKB Web App Search KB Documents Technology used: Adobe Flex / PHP / KB REST API
  • 23. Current Status: Basic Search and Retrieval Live URL: http://tinyurl.com/ykaa9gr
  • 24.
  • 26. Terminate themNext Step: Add more EC2 features, integrate with FutureGrid user database
  • 28.
  • 29. On both real hardware and Virtual machines.
  • 30. We are working to capture common XCat tasks as scripts
  • 31. These scripts can be wrapped as secure services using OGCE’sGFac.
  • 32. Several tasks can be linked together as workflows visually composed with OGCE’sXBaya.
  • 33.
  • 34. Science Gateways Layer Cake User Interfaces Gateway Abstraction Interfaces Web Enabled Desktop Applications Web/Gadget Container Web/Gadget Interfaces Information Services Application Abstractions Application Monitoring User Management Fault Tolerance Gateway Services Provenance & Metadata Management Registry Workflow System Security Auditing & Reporting Resource Middleware Cloud Interfaces SSH & Resource Managers Grid Middleware Compute Resources Local Resources Computational Grids Computational Clouds Color Coding OGCE Gateway Components Complimentary Gateway Components Dependent resource provider components
  • 35. Google Gadget-Based Science Gateways PolarGrid MyOSG LEAD
  • 36. GFac Current & Future Features Apache Axis2 Globus Registry Interface Scheduling Interface Input Handlers Monitoring Interface Campus Resources Data Management Abstraction Fault Tolerance Output Handlers Amazon Eucalyptus Job Management Abstraction Auditing Checkpoint Support Unicore Condor Color Coding Existing Features Planned/Requested Features
  • 37. OGCE Layered Workflow Architecture:Derived from LEAD Workflow System Workflow Interfaces (Design & Definition) XBaya GUI (Composition, Deploying, Steering & Monitoring) Flex/Web Composition Gadget Interface for Input Binding Python BPEL 2.0 Workflow Specification Scufl BPEL 1.0 Java Code Pegasus DAG Workflow Execution & Control Engines Apache ODE Condor DAGMan Dynamic Enactor Jython Interpreter GBPEL Taverna
  • 38. Putting It All Together
  • 39. Software Strategy Focus on gadget container and tools for running science applications on grids and clouds. Provide a tool set that can be used in whole or in part. If you just want GFac, then you can use it without buying an entire framework. Outsource security, information services, data and metadata, advanced job handling, etc to other providers. MyProxy, TG IIS, Globus, Condor, XMC Cat, iRods, etc.
  • 40. Packaging, Building, and Testing All builds are designed to be self contained Use Apache Maven 2.x Download includes everything you need. Portal, Axis Services, and Xregistry all build nightly on the NMI Build and Test facility at UW. Several Linux platforms, Mac PPC, and Mac X86 Java 1.5 Apache JMetertest suite for the portal. Run against your installation Automated tests nightly
  • 41. Next Steps Apache Incubator Project for XBaya, GFac and supporting workflow tools WIYN ODI instrument pipeline and gateway Robert Henschel and Scott Michael are leading overall effort. Suresh and Raminder are working 50% time through early April on technical evaluation of integrating NHPPS software with OGCE New Collaboration: Craig Mattocks, UNC will build a storm surge forecasting gateway Broadly similar to LEAD and SCOOP Archit will be the point of contact Local gateway tutorial in early April Sun Kim’s group, UNC Group, CGB Others welcome Gadget Container additional applications OGCE grid gadgets packaged release, SimpleGrid
  • 42. More Information, Acknowledgements Website: www.collab-ogce.org Blog/RSS Feed: collab-ogce.blogspot.com Email: mpierce@cs.indiana.edu, smarru@cs.indiana.edu Geoffrey Fox, Craig Stewart, and Dennis Gannon have spent years laying the foundation for this work
  • 44. EST Assembly Pipeline Support: ArchitKulshrestha, Chin Hua Kong Collaborator: Qunfeng Dong, UNT
  • 45.
  • 46. Computation is outsourced to Grids (TeraGrid) and Clouds (Amazon)
  • 47. Not provided by in-house clusters.
  • 48. This is an open service, open architecture approach.
  • 49. These codes have very different scheduling requirementshttp://swarm.cgb.indiana.edu
  • 50.
  • 51. TeraGrid’sBigRed, Ranger, and Cobalt: PACE, RepeatMasker
  • 53.
  • 54. UltraScan Science Gateway A biophysics gateway for investigating properties and structure-function relationships of biological macromolecules, nanoparticles, polymers and colloids that are implicated in many diseases, including cancer. High-resolution analysis and modeling of hydrodynamic data from an analytical ultracentrifuge. TeraGrid as a backup spill over resources but still is one of the heaviest user consuming 1.75 million SU’s in 6 months.
  • 55. UltraScan Advanced Support Porting to new architectures and parallel performance enhancements. New workflow implementations, new grid computing and grid middleware support: Reliability problems with WSGram Missing job status Only supports Gram4, needs porting to other middleware Issues with data movement. Need Fault tolerance at all levels. Users decide resources manually, need automated scheduling. Current Architecture
  • 56. UltraScan OGCE Integration Enhance the perl job submission daemon with OGCE Gfac service. Enhance socket and email based job monitoring with OGCE Eventing System Implement and iteratively enhance fault tolerance. Port to Community account usage with gridshib auditing support. Support Unicore to run jobs on other European and Australian resources.
  • 57. OGCE based UltraScan development Architecture Manual Process Quarry Gateway Hosting Machine UltraScan Middleware GFac, Eventing System, Fault Tolerance Europe & Australian Grids
  • 58. Future Grid KB Image Manager Experiment Browser INCA Monitor Downloadable Gadgets
  • 59. Deployment (in future) iGoogle Gadgets Image Management FG INCA MONITOR Experiment Management iPhone Application
  • 61. Why Gadgets? We have redesigned many OGCE components to work as gadgets. Fugang Wang’s Cyberaide JavaScript gives us an API Framework and language independent Client-side HTML, CSS, and JavaScript, not server-side Java Integration and content under user’s control, not portal administrator’s Can be integrated into iGoogle and similar containers. 140,000 published gadgets Joomla, Drupal, Liferay, etc We can potentially provide HUBzero gadgets
  • 62. OGCE based UltraScan development Architecture Manual Process Quarry Gateway Hosting Machine UltraScan Middleware GFac and supporting services Europe & Australian Grids
  • 63. BioDrugScreen Next Steps We want to expand the user generated function process, including the ability for a user to save a function and have multiple functions Interaction between users will be enhanced, allowing them to share their functions and findings.