Your SlideShare is downloading. ×
The Symbiotic Nature of Provenance and Workflow
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The Symbiotic Nature of Provenance and Workflow

92
views

Published on

My web page: http://www.linkedin.com/in/ericstephan …

My web page: http://www.linkedin.com/in/ericstephan
My citations: http://scholar.google.com/citations?hl=en&user=f4bH2esAAAAJ

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
92
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. The Symbiotic Nature of Provenance and Workflow Eric Stephan, Todd Halter Pacific Northwest National Laboratory1
  • 2. The Systems Science Challenge!   Studying complex systems typically has the following characteristics: !  Interdisciplinary problem involving various stakeholders !  Leverage multiple tools, algorithms, data products, and sensors !  Reliant on highly iterative and repetitive techniques !  Steps are difficult to document and are often time committed to memory or notes.!   Solution is to provide: !  ‘plumbing’ to more easily configure and automate integration, calculation, analysis, and visualization !  Provide a historical explanation of what occurred2
  • 3. Active Computer Science Research Areas !   Workflows – plumbing !   Provenance – explanation • Without a historical explanation workflows provide capability, but neglect a documentation trail of what transpired. • Without plumbing provenance is difficult to introduce generically or to support legacy applications3
  • 4. Example Workflow Products !   Creating executable workflows from schematic drawings I.  Al&nas,  O.  Barney,  Z.Cheng,  T.  Critchlow,  B.  Ludaescher,  S.  Parker,  A.  Shoshani,  M.  Vouk,   “Accelera&ng  the  Scien&fic  Explora&on  Process  with  Scien&fic  Workflows”,  In  Journal  of   Physics:  Conference  Series  SciDAC  2006  proceedings.    June  2006.4
  • 5. Example Workflow Products !   Constructing component based MeDICi: Middleware for Data- Intensive Computing analytical pipelines on enterprise service bus technology Gorton  I,  AS  Wynne,  JP  Almquist,  and  J  ChaQerton.  2008.  ”The  MeDICi   Integra&on  Framework:  A  PlaVorm  for  High  Performance  Data  Streaming   Applica&ons.”  In  WICSA  2008.  7th  IEEE/IFIP  Working  Conference  on  So[ware   Architecture,  Feb.  18-­‐22,  2008,  Vancouver,  Canada  ,  pp.  95-­‐104.  IEEE   Computer  Society,  Los  Alamitos,  CA.  doi:10.1109/WICSA.2008.21  5
  • 6. Example of Provenance !   Digital Library, Lineage !   Extensible Open Model- Open Provenance Model Moreau  L,  B  Clifford,  J  Freire,  J  Futrelle,  Y  Gil,  P  Groth,  N  Kwasnikowska,  S  Miles,  P  Missier,  J  Myers,  BA  Plale,  YL   Simmhan,  EG  Stephan,  and  J  Van  den  Bussche.    2010.    "The  Open  Provenance  Model  Core  Specifica&on   (v1.1)  ."    Future  Genera@ons  Computer  Systems.    doi:10.1016/j.future.2010.07.005   !   Semantic web-based Models- Proof Markup Language W3C  Incubator  Group,  hQp://www.w3.org/2005/Incubator/prov/wiki/ W3C_Provenance_Incubator_Group_Wiki  6
  • 7. Examples of Creating Connectivity… !   Workflows !  Event listeners !   Self describing workflow components, flow !   Provenance !  Formally described !   Support for reasoning, transitive closure etc. !   Semantically relevant to provenance consumers.7
  • 8. Existing Deficiencies !   Workflows !  Listeners only reporting syntactic events !   Deluge of atomic transactions !   Inability to convey logical constructs !   E.g. initialization stage !   Lack of support to collect logs from legacy applications !   Provenance !  Collecting naïve provenance – big graph dilemma !   Hardcoded – risk being out of sync with workflow !   Collection without end user requirements8
  • 9. Interoperability Aides !   Applying provenance execution models to workflow listeners !  E.g. Describe Anything DaAPI Wynne  AS,  I  Gorton,  JM  Chase,  and  EG  Stephan.    2009.    MeDICi:  An  Open  PlaEorm  for  Sensor  Integra@on  .    PNNL-­‐18716,  Pacific   Northwest  Na&onal  Laboratory,  Richland,  WA.   !   Incorporating provenance in workflow framework !  Semantic Abstract Workflow (SAW) Leonardo  Salayandia  and  Paulo  Pinheiro  da  Silva.  On  the  Use  of  Seman&c  Abstract  Workflows  Rooted  on  Provenance   Concepts  .PROVENANCE  AND  ANNOTATION  OF  DATA  AND  PROCESSES.  Lecture  Notes  in  Computer  Science,  2010,   Volume  6378/2010,  216-­‐220,  DOI:  10.1007/978-­‐3-­‐642-­‐17819-­‐1_24  9
  • 10. Interoperability Aides !   Advanced storage – !  Grids, Semantic Wikis !   New Provenance Model Abstractions Stephan  EG,  TD  Halter,  and  BD  Ermold.    2010.    "Leveraging  The  Open  Provenance  Model  as  a  Mul&-­‐Tier  Model  for   Global  Climate  Research  ."    In  The  3rd  Interna@onal  Provenance  and  Annota@on  Workshop  (IPAW2010). Gibson  TD,  KL  Schuchardt,  and  EG  Stephan.    2009.    "Applica&on  of  Named  Graphs  Towards  Custom  Provenance   Views."    In  1st  Workshop  on  the  Theory  and  Prac&ce  of  Provenance  (TaPP  09),  p.  Paper  No.  5.    USENIX,  Berkeley,  CA.      10
  • 11. Conclusions !   Good news - Workflow and provenance interoperability is evolving. !   Challenge #1: Recognizing existence of symbiotic relationship between Workflow and Provenance. !   Challenge #2: Finding new ways to harness this relationship to advance systems science research.11
  • 12. Questions? !   Contact: eric.stephan@pnl.gov12