Gsoc airavata


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Gsoc airavata

  1. 1. Apache AiravataGSOC 2013
  2. 2. Hi, I’m Nolram.I’m a computationalphysicist.I run computationalexperiments everydayThis is how typically Irun my experiments
  3. 3. Scientific ApplicationAnother ScientificApplicationFirst I collect myobserved dataAnd then pass data tomy applications & getthe resultThis is starting tobecome a very tiringtask
  4. 4. How can I make thismuch simpler…?Logically, this is howmy life would bemade easier…Is it possible toautomate this flowsequence without myguidance?
  5. 5. Scientists from manydifferent fields face thisproblem everyday.The solution is to use aworkflow-poweredscience gateway tomanage the experimentonline.What is a workflow youask?Well, you just saw one inour previous animation…
  6. 6. We introduce Apache Airavata, a system capable ofcomposing, managing, executing, and monitoringsmall to large scale applications and workflowsWant to see how it works?A Typical Workflow
  7. 7. Apache AiravataI will handover my data & myexperiment details (the workflow)to the Airavata serverThe GatewayAiravata will complete theexperiment & return me the resultsResultsProgress of the experiment… and while I wait forresults, Airavata will notify me withprogress updates of my experiment
  8. 8. Let’s look closely how Airavatamanages workflows.The GatewayResultsExperiment progressApache Airavata
  9. 9. Let’s look closely how Airavatamanages workflows.The GatewayResultsExperiment progress
  10. 10. Airavata main has 4 components…The Gateway1. Workflow InterpreterSteer the workflow execution2. The GFacSteer science app executions & datatransfersWorkflow InterpreterGFacMessage BoxRegistry3. The RegistryDefines the available applications &records all results of experiments4. The Message BoxRecords the progress of the workflowexecution
  11. 11. A Stable API forAiravataApache AiravataEndUsersGatewayDeveloperScientificApplicationComputational Resources
  12. 12. ApplicationRegistrationUIApplicationDeveloperA1Airavata ServiceInterface(wraps client API)Service MapXMLAiravataServerService Mapto AWSDLWeb BasedworkflowcomposerWorkflowDeveloperGet AWSDLPut XWFA2A3W1W2W3Web BasedExperimentBuilderExperimentBuilderGet WI’sE1E2E3W4ShredWorkflowInputsLaunchWorkflowWeb BasedWorkflowMonitorWatch ProgressM1M2Get WorkflowGraphM3MonitorWorkflow
  13. 13. Goal of the project• Design Web-Based interfaces for Airavata:– Application Registration– Workflow Construction– Workflow Execution– Workflow Monitoring• Provide an opportunity for GSoC tounderstand Distributed System in action• Scope for Research and Software Engineeringpapers
  14. 14. Data Model• Application Description– User describes inputs and outputs of theapplication.– Currently this information is captured in ServiceMap Schema.– This schema is stored in Airavata Registry as XML.Also the schema utility generates a applicationservice WSDL from this schema using the AiravataWSDL Generator.
  15. 15. ApplicationRegistrationUIApplicationDeveloperA1Service MapXMLService Mapto AWSDLWeb BasedworkflowcomposerWorkflowDeveloperGet AWSDLA2W1W2XMLAiravataServerAPILaunch & Manage JobsNotify progress of jobor workflow executionReal-TimeMonitoringMessagingSubsystemRegistryExecute & ManageComputationsWorkflowInterpreterApplicationFactory (Gfac)Application DescApplication DescWorkflow
  16. 16. A peek at one of the clusterInterconnectNodes
  17. 17. Scheduling ‘qsub’ batch jobs on the clusterworkernodeworkernodeworkernodeworkernodeworkernodeSGE MASTER nodeQueue-A Queue-B Queue-CASlot1ASlot2BSlot1CSlot1CSlot2CSlot3BSlot1BSlot2BSlot3BSlot1CSlot1CSlot2ASlot1BSlot1CSlot1QueuesPoliciesPrioritiesShare/TicketsResourcesUsers/ProjectsJOB Y JOB ZJOB XJOB UJOB OJOB N
  18. 18. SystemcharacteristicsSystem statusResourcesJob policiesResourcesResource MatchingJOBUserUser policiesGroupsRolesDepartmentsProjectsSelection Scheduling
  19. 19. Simplified Gateway ArchitectureOne time GatewayCommunity SetupCommunity Account Grid Certificateusername, passwordGateway InterfaceGateway ServerCompute ServersGateway AuthenticationJob Submit orFile Transfer requestOutputStep 0Step 1Step 2,3,,
  20. 20. ParamChemBioVLabGridChemDESVLABUltraScanApacheAiravata 1.0NSGParamChemBioVLabCIPRESPOPLARGridChemDES UltraScanVLABApacheAiravata 2.0ApacheAiravata 1.0ApacheAiravata 1.0ApacheAiravata 1.0ApacheAiravata 1.0ApacheAiravata 1.0