A Chemistry-Inspired Workflow ManagementSystem for Scientific Applications in Clouds          7th IEEE International Confe...
Context               • Scientific applications developed as workflows demanding more computational power.                ...
Objectives               • Ensure a workflow execution:                   • Decentralized.                   • Loosely cou...
Chemical Programing Model (I)                               • A program can be seen as a chemical solution:Chemical Progra...
Chemical Programing Model (II)Chemical Programing paradigm                               • Example:                       ...
HOCL-based Workflow System                             6
Chemical Coordination: Workflow Definition                              • Express all data and control dependencies (react...
Chemical Coordination: Generic Rules                              • Independent from any chemical workflow representation....
Chemical Coordination: Workflow Patterns                              • Control flow can be expressed using some generic r...
Architectures• Coordination mechanism built upon HOCL.• Two possible architectures for our workflow system:    • Centraliz...
Centralized Architecture                           • Central node coordinates all data and control flow between the Web se...
Decentralized Architecture (I)                           • Nodes communicating through a shared address space.Chemical Wor...
Decentralized Architecture (III)                           • Multiset, dynamic and decentralized coordination mechanism.  ...
Implementation                 14
Centralized Prototype                           • Service caller                               • Interface with all the co...
Decentralized Prototype                           • Chemical Web Services (ChWS):Chemical Workflow System                 ...
Experiments              17
Experiments (I)                      • Objective: Establish the viability of our chemical workflow engine in comparison wi...
Performance Results                      Experiments (II)
Performance Results                      Experiments (II)
Performance Results                           Results21
Centralized Experiment                      Data and computation intensive workflows.                        • Size and pr...
Decentralized Experiment                      Reduced computation workflows                        • Slightly increment of...
Conclusion          • Chemical model is well featured for decentralized workflow execution.                 Proof of conc...
On-going Work• Implementation of a distributed multiset.• Workflow scheduling in Federated Clouds using the chemical model...
Questions   ?                26
THANKS !           27
Upcoming SlideShare
Loading in …5
×

A Chemistry-Inspired Workflow Management System for Scientific Applications on Clouds

566 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
566
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

A Chemistry-Inspired Workflow Management System for Scientific Applications on Clouds

  1. 1. A Chemistry-Inspired Workflow ManagementSystem for Scientific Applications in Clouds 7th IEEE International Conference on e–Science Stockholm 2011 Hector Fernandez, Cedric Tedeschi and Thierry Priol 00 MOIS 2011
  2. 2. Context • Scientific applications developed as workflows demanding more computational power.  Demand for deployment on Grids or Clouds. • Scientific workflow management systems (WMS):Introduction  Implicit parallelism.  Data-driven coordination.  Support for the execution on Grids. • Examples of Scientific WMS: Taverna, Pegasus, Triana and Kepler. • Requirements of next generation Scientific WMS: • Management of high degree of parallelism and distribution. • No single point of failure. • Scalability. • Dynamicity. 2
  3. 3. Objectives • Ensure a workflow execution: • Decentralized. • Loosely coupled (coordination mechanism).Introduction • Dynamic. • Autonomous. “Nature-inspired metaphors have been shown to be of high interest for service coordination.” [Viroli et al., 2009]. ➔ Evaluate the viability of a nature-inspired scientific workflow system. 3
  4. 4. Chemical Programing Model (I) • A program can be seen as a chemical solution:Chemical Programing paradigm • Data: “floating” molecules in the solution. • Computation: chemical reactions between the molecules. • Implicit parallelism and autonomy of reactions until inertia. • Expression of dynamicity. • Data structure: Multiset (blackboard). • Containing all data molecules. • Reaction rules re-writing the multiset. • Languages: • Gamma (Pioneered model) [Banâtre et al.,1990]. • HOCL ( High-Order model) [Radenac, 2007]. 4
  5. 5. Chemical Programing Model (II)Chemical Programing paradigm • Example: • A reaction rules is written replace-one P by M if C where P is a pattern which matches the required molecule, C is the reaction condition and M the result of the reaction. 5
  6. 6. HOCL-based Workflow System 6
  7. 7. Chemical Coordination: Workflow Definition • Express all data and control dependencies (reaction rules and molecules).Chemical Coordination Model • Molecular composition to express the logic of a workflow. MULTISET 7
  8. 8. Chemical Coordination: Generic Rules • Independent from any chemical workflow representation.Chemical Coordination Model • Used by chemical engines. • Common tasks during a workflow execution: • Service invocation rule. • Control and data transfer rule. 8
  9. 9. Chemical Coordination: Workflow Patterns • Control flow can be expressed using some generic rules.Chemical Coordination Model • Molecular composition of composed generic rules, reactions triggering reactions. Discriminator pattern • More patterns: parallel split, synchronization, exclusive choice, synchronization merge, cancel activity or simple merge. 9
  10. 10. Architectures• Coordination mechanism built upon HOCL.• Two possible architectures for our workflow system: • Centralized. • Decentralized. 10
  11. 11. Centralized Architecture • Central node coordinates all data and control flow between the Web services.Chemical Workflow System • A chemical encapsulation per Web service participating in the workflow. • Multiset as storage space containing the workflow definition. • Chemical engine processing the content of the multiset. 11
  12. 12. Decentralized Architecture (I) • Nodes communicating through a shared address space.Chemical Workflow System • Persistent. • Fault-tolerant. • Workflow executed in parts corresponding with each Web service. • Data and control transfer through this shared space. • Each node is co-responsible of the execution. 12
  13. 13. Decentralized Architecture (III) • Multiset, dynamic and decentralized coordination mechanism. • Acts as a shared address space containing both control and data flows.Chemical Workflow System • ChWSes communicate through the multiset. (reading and writing) • Physically distributed over ChWSes storage spaces. 13
  14. 14. Implementation 14
  15. 15. Centralized Prototype • Service caller • Interface with all the concrete Wses.Chemical Workflow System • Implemented based on Daios framework. • HOCL Interpreter • Central engine. • Multiset • Workflow definition. • Processed by the HOCL Interpreter. 15
  16. 16. Decentralized Prototype • Chemical Web Services (ChWS):Chemical Workflow System • Service caller  Interface with one concrete WS. • Local Multiset  Temporary store space. • HOCL Interpreter  Local workflow engine. • JMS publisher/subscriber  Communication module with the Multiset. • Multiset: • Storage space containing the whole workflow. • Similarities with tuplespaces. • JMS publisher/subscriber  Communication module with the ChWSes. 16
  17. 17. Experiments 17
  18. 18. Experiments (I) • Objective: Establish the viability of our chemical workflow engine in comparison with four WMS. • Four workflow engines: • Kepler 2.0.Performance Results • Taverna Workbench 2.2.0. • Centralized prototype (HOCL Cen.). • Decentralized prototype (HOCL Dec.). • Real scenarios: • Cardiovascular image analysis workflow (CardiacAnalysis) [7]. • Astronomical image mosaics workflow (Montage) [8]. • Bio-informatics workflow (BlastReport) [9]. CardiacAnalysis Montage BlastReport Num. services 6 27 5 Data exchanged High Low Medium Coord. Complex High Medium Low • Experiments conducted on the French research infrastructure Grid5000. 18
  19. 19. Performance Results Experiments (II)
  20. 20. Performance Results Experiments (II)
  21. 21. Performance Results Results21
  22. 22. Centralized Experiment Data and computation intensive workflows. • Size and processing time increment.Performance Results Centralized coordination better for workflows with reduced computation. 22
  23. 23. Decentralized Experiment Reduced computation workflows • Slightly increment of time (network latency).Performance Results Data and computation-intensive workflows show the benefits of a decentralized coordination. 23
  24. 24. Conclusion • Chemical model is well featured for decentralized workflow execution.  Proof of concept of the chemical workflow system.Summary • Our proposal: High-level decentralized coordination mechanism. • Decentralized Architecture:  Chemical web services working as local engines.  Multiset as shared communication space.  A High-order chemical language for workflows. • Concepts for decentralized coordination. • Control and data driven. 24
  25. 25. On-going Work• Implementation of a distributed multiset.• Workflow scheduling in Federated Clouds using the chemical model.• Modelling Agile Service Networks using the chemical choreography coordination model. 25
  26. 26. Questions ? 26
  27. 27. THANKS ! 27

×