Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Csc8710 001 winter2014-mohammed_shahnawazali-ff2687_presentation_2

1,772 views

Published on

CSC8710-001_Winter2014_MohammedShahnawazAli-ff2687_Presentation_2

Published in: Education
  • Be the first to comment

  • Be the first to like this

Csc8710 001 winter2014-mohammed_shahnawazali-ff2687_presentation_2

  1. 1. Combining Data-Flow and Control-Flow For Scientific Workflows CSC 8710-001 – Presentation 2 Mohammed Shahnawaz Ali
  2. 2. Executive Summary • Data-centric scientific workflows modeled as dataflow process networks • Establish a generic framework for embedding control-flow intensive tasks • Make scientific workflows more robust and reusable 2/19/2014 CSC 8710 - Presentation 2 2
  3. 3. Objective & Structure Describe:  Scientific Workflow Systems – Usage & Models  Actor Oriented Workflow  Building Blocks  Design  Extensions  A 3-tier architecture framework  Design  Usage  Closing Notes  Next Steps 2/19/2014 CSC 8710 - Presentation 2 3
  4. 4. Scientific Workflow Systems – Usage • Used to construct and execute complex data-centric scientific analyses • Requires bringing together – data retrieval, computation, visualization • Support end-to-end workflow management 2/19/2014 CSC 8710 - Presentation 2 4
  5. 5. Scientific Workflow Systems – Models • Directed Acyclic Graph with arcs for scheduling dependencies between jobs. • Dataflow Process Networks with built-in support for stream based and concurrent execution o Efficient analysis o Simple and intuitive 2/19/2014 CSC 8710 - Presentation 2 5
  6. 6. Actor Oriented Workflow – Building Block Building blocks of actor-oriented modeling + design: • Actors: Workflow components wired together by ports o Composite: Encapsulate sub-workflows • Director: Overall execution and component interaction, behavioral polymorphism • Port: Input and Output 2/19/2014 CSC 8710 - Presentation 2 6
  7. 7. Actor Oriented Model – Design Actor oriented workflow graph, W = ‹A,D› [A: Actors, D: Dataflow connections] Signature of Actor, ∑A = in(A) → out(A) Dataflow connection, d Є D, is a directed hyperedge d = ‹o,i› [o: output, i: input] has: 1. Merge step 2. Copy step 3. Delivery step 2/19/2014 CSC 8710 - Presentation 2 7
  8. 8. Actor Oriented Model – Design (Cont’d) • A composite actor Aw encapsulates sub-workflow W. • The ports of Aw consists of set of W ports. • Hierarchical workflow contains at least one composite actor with any level of nesting 2/19/2014 CSC 8710 - Presentation 2 8
  9. 9. Actor Oriented Model – Extensions Two extensions to actor oriented modeling: • Frames: Abstraction that denotes a set of alternative actor implementations with similar functionality. • Templates: Abstraction for a set of workflows that specifies the behavior of the workflow it represents. 2/19/2014 CSC 8710 - Presentation 2 9
  10. 10. Actor Oriented Model – Frames • Used as abstractions for a family of components with similar function. • Placeholders for components that will be instantiated and specialized later. • Has input, output, and parameter ports, structural types, and semantic types – frame signature 2/19/2014 CSC 8710 - Presentation 2 10
  11. 11. Actor Oriented Model – Frames (cont’d) • F[C] in ports(F) X ports(C) • The embedded component may: o introduce new ports o not use all the ports • Parameter ports can also be connected to input ports and vice versa 2/19/2014 CSC 8710 - Presentation 2 11
  12. 12. Actor Oriented Model – Frames (cont’d) • Embedding F[C] is well-formed if the input and output port directions are observed. • A well-formed embedding is structurally well-typed and/or semantically well-typed. • The typing rules can be relaxed when the frames occur within a workflow. • Provides natural mechanism to execute associated actors in parallel. 2/19/2014 CSC 8710 - Presentation 2 12
  13. 13. Actor Oriented Model – Workflow Templates Workflow Template • Specifies the behavior of the workflows it represents. • ∑T : in(T) → out(T) • Includes an “inner” workflow graph WT with some of the components as frames. 2/19/2014 CSC 8710 - Presentation 2 13
  14. 14. Actor Oriented Model – Workflow Templates (cont’d) Workflow Template • T represents a partial workflow specification. • Frames can be independently specialized by embedded components • Resulting embedding is : • either a concrete, executable workflow • or a template 2/19/2014 CSC 8710 - Presentation 2 14
  15. 15. Actor Oriented Model – Transducer Templates Transducer Template • The template T can constrain by providing one or more directors. • FST director inscribed indicates executing the workflow graph WT as a finite state transducer. • The director dictates: 1. Execution model 2. Constraints on the graph. 2/19/2014 CSC 8710 - Presentation 2 15
  16. 16. Generic Control-Flow Component Pattern – Objective Objective: Structure frames and templates that can be executed using, 1. Alternative control behavior 2. Alternative task implementation 2/19/2014 CSC 8710 - Presentation 2 16
  17. 17. Generic Control-Flow Component Pattern – Design Design: Consists of three tiers/levels: 1. Level 1: • A frame within a dataflow graph and denotes a particular task. • Can be embedded with finite state transducer templates 2. Level 2: • Transducer templates for control-flow behavior. • Has one or more state frames. • Offers a more natural, intuitive, succinct language 2/19/2014 CSC 8710 - Presentation 2 17
  18. 18. Generic Control-Flow Component Pattern – Design (cont’d) Design: 3. Level 3: • State Frames that can be embedded in a particular task implementation. An FST is a tuple M = ‹I, O, Q, q0, T› 2/19/2014 CSC 8710 - Presentation 2 18
  19. 19. Generic Control-Flow Component Pattern – Usage Usage: Implementation enables workflow designers to configure both the behavior and underlying implementation. Specifically, a workflow designer can, 1. Insert into a workflow generic component. 2. Select an available transducer template behavior. 3. Select task implementations for the state frames and templates. 2/19/2014 CSC 8710 - Presentation 2 19
  20. 20. Closing Notes • Scientific workflows are primarily dataflow oriented, certain workflows can be control-intensive • The generic framework describes how to support structured embedding of generic control-flow components within data process networks. • Frames and templates can be used to develop robust workflows via reusable control-intensive subtasks. 2/19/2014 CSC 8710 - Presentation 2 20
  21. 21. Next Steps • • • Fully integrate frames and templates as first class modeling constructs. Develop additional transducer templates and lower level implementation components. Explore mechanisms for easily combining transducer templates. 2/19/2014 CSC 8710 - Presentation 2 21
  22. 22. Acknowledgements • • • • Shawn Bowers – UC Davis Genome Center, University Of California, Davis. Bertram Ludascher – UC Davis Genome Center, University Of California, Davis. Anner H.H. Ngu – Department of Computer Science, Texas State University. Terrence Crtichlow – Center for Applied Scientific Computing, Lawrence Livermore National Laboratory. References • G. Alonso and C. Mohan. Workflow management systems: The next generation of distributed processing tools. In Advanced Transaction Models and Architectures. • C. Berkley, S. Bowers, M. Jones, B. Lud¨ascher, M. Schildhauer, and J. Tao. Incorporating semantics in scientific workflow authoring. In Proc. of the Intl. Conf. on Scientific and Statistical Database Management (SSDBM). • V. Bhat, S. Klasky, S. Atchley, M. Beck, D. McCune, and M. Parashar. High performance threaded data streaming for large scale simulations. In Proc. of the IEEE/ACM Intl.Workshop on Grid Computing (GRID’04). 2/19/2014 CSC 8710 - Presentation 2 22
  23. 23. Thank You 2/19/2014 CSC 8710 - Presentation 2 23

×