Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Taverna workflows: provenance and reproducibility - STFC/NERC workshop 2013


Published on

Slides on Taverna from the talk given at STFC/NERC workshop "Workflow approaches to investigation of biological complexity", 15-16 October 2013.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Taverna workflows: provenance and reproducibility - STFC/NERC workshop 2013

  1. 1. Taverna workflows: provenance and reproducibility Aleksandra Pawlik The University of Manchester Workflow approaches to investigation of biological complexity STFC/NERC Workshop 15-16 October 2013
  2. 2. Workflows for improvement Workflows are more than just pipelines… Scaling up automated execution Bringing together distributed and continually changing resources Dealing with different standards, interfaces and implementation Support for repeatable analysis
  3. 3. Taverna Engine Execution Workflows in Scufl2  Functional dataflow, simple control flows, implicit  iteration Linking services and tools  Different data resources and formats  “In Workflow Programming” (eg. Beanshell scripting)  Provenance collection: W3C PROV-O, OPM  Plug-in Framework     Infrastructures: Web Services (SOAP, REST), Grid, HPC Common Tools: Excel Spreadsheets, Google Refine, R OAuth security plug-in
  4. 4. Taverna Workbench • Customizable for domains (eg. expose services only for biodiversity) • Desktop application • Intermediate results views • Plug-in framework List of services Workflow engine to run workflows Construct and visualise workflows
  5. 5. Taverna User Spectrum Taverna Concept Knowledge Workflow Engineer Workbench Computational Scientist Workbench Components Lite Domain Domain Scientist (Workflow User) Domain-Specific Website / Tool / Portal Player High Workflow Visibility Low
  6. 6. reuse   Right apps, right users Commodity apps:          Web. Spreadsheets. R. Customisation Mixed workflow / scripting Deployment / Portability  Apps Apps Web based / desktop Virtualised deployments Cloud hosted service A cloud-enabled local host Local ownership Capability building Workflow Workflow WFMS WFMS middleware middleware Infrastructure Infrastructure Domain/task specific apps that incorporate (an ecosystem of) workflows. Integrate Parameterised, integrative, multi-step (data) pipelines, analytics, computational protocols. Can be repetitively reused. Support design, config. and execution of workflows. manage utility actions for data, logging, security, compute, error. Shield incompatibilities & complexity. Legacy, others and your own software, datasets, services, codes, and platforms. Optimise and manage use of computing infrastructure.
  7. 7. Reuse and Reproducibility
  8. 8. ~6,000 members over 300 groups, over 3,000 workflows
  9. 9. Taverna Components Workflow Blocks made of a workflow  Well described  Well behaved  Well looked after  Agreed fail  Agreed formats in and out  Agreed provenance Deposited in myExperiment Grouped into families
  10. 10. Provenance: how did you do it?  The link between computation and results d1  Reporting at different scales/ levels d2 S1 S0 S1 w -> Using Provenance d1' S0  Collecting d2 z w S2 S'2 y y' S4 S4 df df' (i) Trace A (ii) Trace B PDIFF: comparing provenance traces to diagnose divergence across experimental results [Woodman et al, 2011]
  11. 11. Research Objects bundles and relates digital resources of a scientific experiment or investigation using standard mechanisms
  12. 12. Taverna in Galaxy Wrap as Tool Tools Workflow in Upload Galaxy execution Taverna server
  13. 13. The Taverna Suite of Tools Workflow Repository User Interfaces Workbench Service Catalogue Workflow Engine Workflow Provenance Activity and Service Plug-in Manager Taverna Lite Workflow Server Web Portals / Gateways Client User Interfaces Third Party Tools Player Virtual Machine Workflow Components Command Interaction Line Server Prog APIs
  14. 14. Sustainability and user support Freely available Open source Current version 2.4 80,000+ downloads across version Windows/Mac OS X/ Linux/Unix Tutorials and Workshops Active user forum & support
  15. 15. Taverna in other projects BioDiversity Virtual e-Laboratory SCAPE Wf4Ever VPH-Share HELIO iPlant Collaborative www.iplantcollaborative.or g HELIO Pacific Northwest National Laboratory KBase Scientific Workflows and Provenance Working Group SHIWA
  16. 16. Products Methods Data-centric Computation Scientific workflows over Distributed Cyber-Infrastructure. Data sharing libraries and catalogues for all types of scientific artefacts and all types of scientists. Knowledge Management Metadata, semantics digital exchange, preservation, publishing Software Engineering Software sustainability, software and data policy, training
  17. 17. For more information  Taverna   myExperiment   myGrid 