1. A Chemistry-Inspired Workflow Management
System for Scientific Applications in Clouds
7th IEEE International Conference on e–Science
Stockholm 2011
Hector Fernandez, Cedric Tedeschi and Thierry Priol 00 MOIS 2011

2. Context
• Scientific applications developed as workflows demanding more computational power.
 Demand for deployment on Grids or Clouds.
• Scientific workflow management systems (WMS):
Introduction
 Implicit parallelism.
 Data-driven coordination.
 Support for the execution on Grids.
• Examples of Scientific WMS: Taverna, Pegasus, Triana and Kepler.
• Requirements of next generation Scientific WMS:
• Management of high degree of parallelism and distribution.
• No single point of failure.
• Scalability.
• Dynamicity.
2

3. Objectives
• Ensure a workflow execution:
• Decentralized.
• Loosely coupled (coordination mechanism).
Introduction
• Dynamic.
• Autonomous.
“Nature-inspired metaphors have been shown to be of high interest
for service coordination.”
[Viroli et al., 2009].
➔
Evaluate the viability of a nature-inspired scientific workflow system.
3

4. Chemical Programing Model (I)
• A program can be seen as a chemical solution:
Chemical Programing paradigm
• Data: “floating” molecules in the solution.
• Computation: chemical reactions between the molecules.
• Implicit parallelism and autonomy of reactions until inertia.
• Expression of dynamicity.
• Data structure: Multiset (blackboard).
• Containing all data molecules.
• Reaction rules re-writing the multiset.
• Languages:
• Gamma (Pioneered model) [Banâtre et al.,1990].
• HOCL ( High-Order model) [Radenac, 2007].
4

5. Chemical Programing Model (II)
Chemical Programing paradigm
• Example:
• A reaction rules is written
replace-one P by M if C
where P is a pattern which matches the required molecule, C is the reaction
condition and M the result of the reaction.
5

6. HOCL-based Workflow System
6

7. Chemical Coordination: Workflow Definition
• Express all data and control dependencies (reaction rules and molecules).
Chemical Coordination Model
• Molecular composition to express the logic of a workflow.
MULTISET
7

8. Chemical Coordination: Generic Rules
• Independent from any chemical workflow representation.
Chemical Coordination Model
• Used by chemical engines.
• Common tasks during a workflow execution:
• Service invocation rule.
• Control and data transfer rule.
8

9. Chemical Coordination: Workflow Patterns
• Control flow can be expressed using some generic rules.
Chemical Coordination Model
• Molecular composition of composed generic rules, reactions triggering reactions.
Discriminator pattern
• More patterns: parallel split, synchronization, exclusive choice, synchronization merge, cancel
activity or simple merge.
9

10. Architectures
• Coordination mechanism built upon HOCL.
• Two possible architectures for our workflow system:
• Centralized.
• Decentralized.
10

11. Centralized Architecture
• Central node coordinates all data and control flow between the Web services.
Chemical Workflow System
• A chemical encapsulation per Web service participating in the workflow.
• Multiset as storage space containing the workflow definition.
• Chemical engine processing the content of the multiset.
11

12. Decentralized Architecture (I)
• Nodes communicating through a shared address space.
Chemical Workflow System
• Persistent.
• Fault-tolerant.
• Workflow executed in parts corresponding with each Web service.
• Data and control transfer through this shared space.
• Each node is co-responsible of the execution.
12

13. Decentralized Architecture (III)
• Multiset, dynamic and decentralized coordination mechanism.
• Acts as a shared address space containing both control and data flows.
Chemical Workflow System
• ChWSes communicate through the multiset. (reading and writing)
• Physically distributed over ChWSes storage spaces.
13

14. Implementation
14

15. Centralized Prototype
• Service caller
• Interface with all the concrete Wses.
Chemical Workflow System
• Implemented based on Daios framework.
• HOCL Interpreter
• Central engine.
• Multiset
• Workflow definition.
• Processed by the HOCL Interpreter.
15

16. Decentralized Prototype
• Chemical Web Services (ChWS):
Chemical Workflow System
• Service caller
 Interface with one concrete WS.
• Local Multiset
 Temporary store space.
• HOCL Interpreter
 Local workflow engine.
• JMS publisher/subscriber
 Communication module with the Multiset.
• Multiset:
• Storage space containing the whole workflow.
• Similarities with tuplespaces.
• JMS publisher/subscriber
 Communication module with the ChWSes.
16

17. Experiments
17

18. Experiments (I)
• Objective: Establish the viability of our chemical workflow engine in comparison with four WMS.
• Four workflow engines:
• Kepler 2.0.
Performance Results
• Taverna Workbench 2.2.0.
• Centralized prototype (HOCL Cen.).
• Decentralized prototype (HOCL Dec.).
• Real scenarios:
• Cardiovascular image analysis workflow (CardiacAnalysis) [7].
• Astronomical image mosaics workflow (Montage) [8].
• Bio-informatics workflow (BlastReport) [9].
CardiacAnalysis Montage BlastReport
Num. services 6 27 5
Data exchanged High Low Medium
Coord. Complex High Medium Low
• Experiments conducted on the French research infrastructure Grid'5000.
18

19. Performance Results
Experiments (II)

20. Performance Results
Experiments (II)

21. Performance Results
Results
21

22. Centralized Experiment
Data and computation intensive workflows.
• Size and processing time increment.
Performance Results
Centralized coordination better for workflows with reduced computation.
22

23. Decentralized Experiment
Reduced computation workflows
• Slightly increment of time (network latency).
Performance Results
Data and computation-intensive workflows show the benefits of a decentralized
coordination.
23

24. Conclusion
• Chemical model is well featured for decentralized workflow execution.
 Proof of concept of the chemical workflow system.
Summary
• Our proposal: High-level decentralized coordination mechanism.
• Decentralized Architecture:
 Chemical web services working as local engines.
 Multiset as shared communication space.
 A High-order chemical language for workflows.
• Concepts for decentralized coordination.
• Control and data driven.
24

25. On-going Work
• Implementation of a distributed multiset.
• Workflow scheduling in Federated Clouds using the chemical model.
• Modelling Agile Service Networks using the chemical choreography
coordination model.
25

26. Questions ?
26

27. THANKS !
27