CSC8710-001_Winter2014_MohammedShahnawazAli-ff2687_Presentation_2

1. Combining Data-Flow and Control-Flow
For Scientific Workflows
CSC 8710-001 – Presentation 2
Mohammed Shahnawaz Ali

2. Executive Summary
• Data-centric scientific workflows modeled as dataflow process
networks
• Establish a generic framework for embedding control-flow
intensive tasks
• Make scientific workflows more robust and reusable
2/19/2014
CSC 8710 - Presentation 2
2

3. Objective & Structure
Describe:
 Scientific Workflow Systems – Usage & Models
 Actor Oriented Workflow
 Building Blocks
 Design
 Extensions
 A 3-tier architecture framework
 Design
 Usage
 Closing Notes
 Next Steps
2/19/2014
CSC 8710 - Presentation 2
3

4. Scientific Workflow Systems – Usage
• Used to construct and execute complex
data-centric scientific analyses
• Requires bringing together – data retrieval,
computation, visualization
• Support end-to-end workflow management
2/19/2014
CSC 8710 - Presentation 2
4

5. Scientific Workflow Systems – Models
• Directed Acyclic Graph with arcs for
scheduling dependencies between jobs.
• Dataflow Process Networks with built-in
support for stream based and concurrent
execution
o Efficient analysis
o Simple and intuitive
2/19/2014
CSC 8710 - Presentation 2
5

6. Actor Oriented Workflow – Building Block
Building blocks of actor-oriented modeling + design:
• Actors: Workflow components wired together
by ports
o Composite: Encapsulate sub-workflows
• Director: Overall execution and component
interaction, behavioral polymorphism
• Port: Input and Output
2/19/2014
CSC 8710 - Presentation 2
6

7. Actor Oriented Model – Design
Actor oriented workflow graph, W = ‹A,D› [A: Actors, D: Dataflow connections]
Signature of Actor, ∑A = in(A) → out(A)
Dataflow connection, d Є D, is a directed hyperedge d = ‹o,i› [o: output, i: input] has:
1. Merge step
2. Copy step
3. Delivery step
2/19/2014
CSC 8710 - Presentation 2
7

8. Actor Oriented Model – Design (Cont’d)
• A composite actor Aw encapsulates sub-workflow W.
• The ports of Aw consists of set of W ports.
• Hierarchical workflow contains at least one composite actor with any level of
nesting
2/19/2014
CSC 8710 - Presentation 2
8

9. Actor Oriented Model – Extensions
Two extensions to actor oriented modeling:
• Frames: Abstraction that denotes a set of alternative
actor implementations with similar
functionality.
• Templates: Abstraction for a set of workflows that
specifies the behavior of the workflow
it represents.
2/19/2014
CSC 8710 - Presentation 2
9

10. Actor Oriented Model – Frames
• Used as abstractions for a family of components with similar function.
• Placeholders for components that will be instantiated and specialized later.
• Has input, output, and parameter ports, structural types, and
semantic types – frame signature
2/19/2014
CSC 8710 - Presentation 2
10

11. Actor Oriented Model – Frames (cont’d)
• F[C] in ports(F) X ports(C)
• The embedded component may:
o introduce new ports
o not use all the ports
• Parameter ports can also be connected to input ports and vice versa
2/19/2014
CSC 8710 - Presentation 2
11

12. Actor Oriented Model – Frames (cont’d)
• Embedding F[C] is well-formed if the input and output port directions are
observed.
• A well-formed embedding is structurally well-typed and/or semantically well-typed.
• The typing rules can be relaxed when the frames occur within a
workflow.
• Provides natural mechanism to execute associated actors in parallel.
2/19/2014
CSC 8710 - Presentation 2
12

13. Actor Oriented Model – Workflow Templates
Workflow Template
• Specifies the behavior of the workflows it represents.
• ∑T : in(T) → out(T)
• Includes an “inner” workflow graph WT with some of the
components as frames.
2/19/2014
CSC 8710 - Presentation 2
13

14. Actor Oriented Model – Workflow Templates (cont’d)
Workflow Template
• T represents a partial workflow specification.
• Frames can be independently specialized by embedded components
• Resulting embedding is :
• either a concrete, executable workflow
• or a template
2/19/2014
CSC 8710 - Presentation 2
14

15. Actor Oriented Model – Transducer Templates
Transducer Template
• The template T can constrain by providing one or more directors.
• FST director inscribed indicates executing the workflow graph WT as a finite state
transducer.
• The director dictates:
1. Execution model
2. Constraints on the graph.
2/19/2014
CSC 8710 - Presentation 2
15

16. Generic Control-Flow Component Pattern – Objective
Objective:
Structure frames and templates that can be executed
using,
1. Alternative control behavior
2. Alternative task implementation
2/19/2014
CSC 8710 - Presentation 2
16

17. Generic Control-Flow Component Pattern – Design
Design:
Consists of three tiers/levels:
1. Level 1:
• A frame within a dataflow graph and denotes a
particular task.
• Can be embedded with finite state transducer
templates
2. Level 2:
• Transducer templates for control-flow behavior.
• Has one or more state frames.
• Offers a more natural, intuitive, succinct
language
2/19/2014
CSC 8710 - Presentation 2
17

18. Generic Control-Flow Component Pattern – Design (cont’d)
Design:
3. Level 3:
• State Frames that can be embedded in a
particular task implementation.
An FST is a tuple M = ‹I, O, Q, q0, T›
2/19/2014
CSC 8710 - Presentation 2
18

19. Generic Control-Flow Component Pattern – Usage
Usage:
Implementation enables workflow designers to configure
both the behavior and underlying implementation.
Specifically, a workflow designer can,
1. Insert into a workflow generic component.
2. Select an available transducer template behavior.
3. Select task implementations for the state frames
and templates.
2/19/2014
CSC 8710 - Presentation 2
19

20. Closing Notes
• Scientific workflows are primarily dataflow oriented,
certain workflows can be control-intensive
• The generic framework describes how to support
structured embedding of generic control-flow
components within data process networks.
• Frames and templates can be used to develop robust
workflows via reusable control-intensive subtasks.
2/19/2014
CSC 8710 - Presentation 2
20

21. Next Steps
•
•
•
Fully integrate frames and templates as first class
modeling constructs.
Develop additional transducer templates and lower
level implementation components.
Explore mechanisms for easily combining transducer
templates.
2/19/2014
CSC 8710 - Presentation 2
21

22. Acknowledgements
•
•
•
•
Shawn Bowers – UC Davis Genome Center, University Of California, Davis.
Bertram Ludascher – UC Davis Genome Center, University Of California, Davis.
Anner H.H. Ngu – Department of Computer Science, Texas State University.
Terrence Crtichlow – Center for Applied Scientific Computing, Lawrence Livermore
National Laboratory.
References
• G. Alonso and C. Mohan. Workflow management systems: The next generation of
distributed processing tools. In Advanced Transaction Models and Architectures.
• C. Berkley, S. Bowers, M. Jones, B. Lud¨ascher, M. Schildhauer, and J. Tao.
Incorporating semantics in scientific workflow authoring. In Proc. of the Intl. Conf.
on Scientific and Statistical Database Management (SSDBM).
• V. Bhat, S. Klasky, S. Atchley, M. Beck, D. McCune, and M. Parashar. High
performance threaded data streaming for large scale simulations. In
Proc. of the IEEE/ACM Intl.Workshop on Grid Computing (GRID’04).
2/19/2014
CSC 8710 - Presentation 2
22

23. Thank You
2/19/2014
CSC 8710 - Presentation 2
23