The Symbiotic Nature of Provenance and                     Workflow    Eric Stephan, Todd Halter    Pacific Northwest Nati...
The Systems Science Challenge!   Studying complex systems typically has the    following characteristics:     !    Interdi...
Active Computer Science Research Areas    !   Workflows – plumbing    !   Provenance – explanation    • Without a historic...
Example Workflow Products    !   Creating executable workflows    from schematic drawings    I.	  Al&nas,	  O.	  Barney,	 ...
Example Workflow Products    !   Constructing component based                                                             ...
Example of Provenance    !   Digital Library, Lineage    !   Extensible Open Model- Open Provenance Model       Moreau	  L...
Examples of Creating Connectivity…    !   Workflows       !   Event listeners       !   Self describing workflow component...
Existing Deficiencies    !   Workflows       !   Listeners only reporting syntactic events             !   Deluge of atomi...
Interoperability Aides    !   Applying provenance execution models to workflow        listeners       !       E.g. Describ...
Interoperability Aides     !   Advanced storage –               !       Grids, Semantic Wikis     !   New Provenance Model...
Conclusions     !   Good news - Workflow and provenance interoperability is         evolving.     !   Challenge #1: Recogn...
Questions?     !   Contact: eric.stephan@pnl.gov12
Upcoming SlideShare
Loading in...5
×

The Symbiotic Nature of Provenance and Workflow

137
-1

Published on

My web page: http://www.linkedin.com/in/ericstephan
My citations: http://scholar.google.com/citations?hl=en&user=f4bH2esAAAAJ

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
137
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The Symbiotic Nature of Provenance and Workflow

  1. 1. The Symbiotic Nature of Provenance and Workflow Eric Stephan, Todd Halter Pacific Northwest National Laboratory1
  2. 2. The Systems Science Challenge!   Studying complex systems typically has the following characteristics: !  Interdisciplinary problem involving various stakeholders !  Leverage multiple tools, algorithms, data products, and sensors !  Reliant on highly iterative and repetitive techniques !  Steps are difficult to document and are often time committed to memory or notes.!   Solution is to provide: !  ‘plumbing’ to more easily configure and automate integration, calculation, analysis, and visualization !  Provide a historical explanation of what occurred2
  3. 3. Active Computer Science Research Areas !   Workflows – plumbing !   Provenance – explanation • Without a historical explanation workflows provide capability, but neglect a documentation trail of what transpired. • Without plumbing provenance is difficult to introduce generically or to support legacy applications3
  4. 4. Example Workflow Products !   Creating executable workflows from schematic drawings I.  Al&nas,  O.  Barney,  Z.Cheng,  T.  Critchlow,  B.  Ludaescher,  S.  Parker,  A.  Shoshani,  M.  Vouk,   “Accelera&ng  the  Scien&fic  Explora&on  Process  with  Scien&fic  Workflows”,  In  Journal  of   Physics:  Conference  Series  SciDAC  2006  proceedings.    June  2006.4
  5. 5. Example Workflow Products !   Constructing component based MeDICi: Middleware for Data- Intensive Computing analytical pipelines on enterprise service bus technology Gorton  I,  AS  Wynne,  JP  Almquist,  and  J  ChaQerton.  2008.  ”The  MeDICi   Integra&on  Framework:  A  PlaVorm  for  High  Performance  Data  Streaming   Applica&ons.”  In  WICSA  2008.  7th  IEEE/IFIP  Working  Conference  on  So[ware   Architecture,  Feb.  18-­‐22,  2008,  Vancouver,  Canada  ,  pp.  95-­‐104.  IEEE   Computer  Society,  Los  Alamitos,  CA.  doi:10.1109/WICSA.2008.21  5
  6. 6. Example of Provenance !   Digital Library, Lineage !   Extensible Open Model- Open Provenance Model Moreau  L,  B  Clifford,  J  Freire,  J  Futrelle,  Y  Gil,  P  Groth,  N  Kwasnikowska,  S  Miles,  P  Missier,  J  Myers,  BA  Plale,  YL   Simmhan,  EG  Stephan,  and  J  Van  den  Bussche.    2010.    "The  Open  Provenance  Model  Core  Specifica&on   (v1.1)  ."    Future  Genera@ons  Computer  Systems.    doi:10.1016/j.future.2010.07.005   !   Semantic web-based Models- Proof Markup Language W3C  Incubator  Group,  hQp://www.w3.org/2005/Incubator/prov/wiki/ W3C_Provenance_Incubator_Group_Wiki  6
  7. 7. Examples of Creating Connectivity… !   Workflows !  Event listeners !   Self describing workflow components, flow !   Provenance !  Formally described !   Support for reasoning, transitive closure etc. !   Semantically relevant to provenance consumers.7
  8. 8. Existing Deficiencies !   Workflows !  Listeners only reporting syntactic events !   Deluge of atomic transactions !   Inability to convey logical constructs !   E.g. initialization stage !   Lack of support to collect logs from legacy applications !   Provenance !  Collecting naïve provenance – big graph dilemma !   Hardcoded – risk being out of sync with workflow !   Collection without end user requirements8
  9. 9. Interoperability Aides !   Applying provenance execution models to workflow listeners !  E.g. Describe Anything DaAPI Wynne  AS,  I  Gorton,  JM  Chase,  and  EG  Stephan.    2009.    MeDICi:  An  Open  PlaEorm  for  Sensor  Integra@on  .    PNNL-­‐18716,  Pacific   Northwest  Na&onal  Laboratory,  Richland,  WA.   !   Incorporating provenance in workflow framework !  Semantic Abstract Workflow (SAW) Leonardo  Salayandia  and  Paulo  Pinheiro  da  Silva.  On  the  Use  of  Seman&c  Abstract  Workflows  Rooted  on  Provenance   Concepts  .PROVENANCE  AND  ANNOTATION  OF  DATA  AND  PROCESSES.  Lecture  Notes  in  Computer  Science,  2010,   Volume  6378/2010,  216-­‐220,  DOI:  10.1007/978-­‐3-­‐642-­‐17819-­‐1_24  9
  10. 10. Interoperability Aides !   Advanced storage – !  Grids, Semantic Wikis !   New Provenance Model Abstractions Stephan  EG,  TD  Halter,  and  BD  Ermold.    2010.    "Leveraging  The  Open  Provenance  Model  as  a  Mul&-­‐Tier  Model  for   Global  Climate  Research  ."    In  The  3rd  Interna@onal  Provenance  and  Annota@on  Workshop  (IPAW2010). Gibson  TD,  KL  Schuchardt,  and  EG  Stephan.    2009.    "Applica&on  of  Named  Graphs  Towards  Custom  Provenance   Views."    In  1st  Workshop  on  the  Theory  and  Prac&ce  of  Provenance  (TaPP  09),  p.  Paper  No.  5.    USENIX,  Berkeley,  CA.      10
  11. 11. Conclusions !   Good news - Workflow and provenance interoperability is evolving. !   Challenge #1: Recognizing existence of symbiotic relationship between Workflow and Provenance. !   Challenge #2: Finding new ways to harness this relationship to advance systems science research.11
  12. 12. Questions? !   Contact: eric.stephan@pnl.gov12

×