Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2014-06-03-Taverna-IS-ENES2

514 views

Published on

Robert Haines, Stian Soiland-Reyes: Taverna
2014-06-03 IS-ENES2 workshop on workflows, Hamburg,

Published in: Technology
  • Be the first to comment

  • Be the first to like this

2014-06-03-Taverna-IS-ENES2

  1. 1. Taverna Robert Haines, Stian Soiland-Reyes myGrid, University of Manchester rhaines@manchester.ac.uk http://orcid.org/0000-0002-9538-7919 IS-ENES2 workshop on workflows, Hamburg, 2014-06-03 http://www.taverna.org.uk/ http://www.mygrid.org.uk/ This work is licensed under a Creative Commons Attribution 3.0 Unported License
  2. 2. Taverna in Context • Comprehensive Scientific Workflow Management System + auxiliary tools/repositories • Based at Manchester with multiple contributions and collaborations • Releases: Three major; numerous rolling intermediate. First release 2004. • Downloads: 90,000+ cumulative; 1000 in first month per intermediate release; user audit for May 2013 had 900+ unique addresses use Taverna • Users: ~380 sites and institutions have used or use Taverna • Support: mailing list, community list and Jira
  3. 3. Taverna workflows • Sophisticated analysis pipelines • A set of services to analyze or manage data (local or remote) • Workflows run through the workbench or via a server • Automation of data flow through services • Control of service invocation • Iteration over data sets • Provenance collection • Extensible and open source
  4. 4. Taverna workflows • Dataflow – Graphically connect data between drag-and- dropped services • Service types – REST, SOAP, Command Line, web interactions, scripts (R, Python, Beanshell) – Domain-specific plugins – Your tool?
  5. 5. Taverna workflows • Nested workflows • Components – Reusable and inter-compatible workflow fragments – Grouped into families – Semantically annotated – Curated
  6. 6. Application Runtime Middleware Resources/Codes/Services Infrastructures Repositories Execution Activity Plug-ins Application Runtime Middleware Resources/Codes/Services Platforms Repositories Registries
  7. 7. Application Runtime Middleware Resources/Codes/Services Infrastructures Repositories Execution Activity Plug-ins Application Runtime Middleware Resources/Codes/Services Platforms Repositories Taverna Desktop Workbench Taverna Online Web Tool Portals and Applications Engine Server Player Cmd line Provenance Third Party Servers BioSTIF Workflows & workflow components PROV, OPM Data Provenance Registries
  8. 8. Workflow Clients for People Technical Computational Scientist Domain Scientist Workbench Components Workflow Visibility Concept KnowledgeTaverna Domain High Low Java library Ruby Gem PlayerCmdLine
  9. 9. Simulation characteristics • Platform • Data • Incorporating codes/services • Scale • Parameter and data sweeps • Interacting • Reporting
  10. 10. Biodiversity marine monitoring and health assessment ecological niche modelling Data Intensive Science Collaborative Science Pilumnus hirtellusEnclosed sea problem (Ready et al., 2010) Sarah Bourlat
  11. 11. Data discovery Data assembly, cleaning, and refinement Ecological Niche Modeling Statistical analysis Analytical cycle Data collection Insights Scholarly Communication & Reporting
  12. 12. www.biovel.eu Ecological Niche Modeling Workflow (ENM)
  13. 13. data configuration parameters steps Data and Parameter Sweeps
  14. 14. Patient Pseudoidentifier (PID) Demographics Height Weight Vital Signs Heart Rate Blood Pressure Flow Rate Transient Pressure Aneurysm Properties Tissue Properties Wall Thickness Risk Factors Medical Images Medications Patients PatientAvatar DiseaseSimulation Workflow SystemicFactors GeneExpressionProfile Patient Pseudoidentifier (PID) Demographics Height Weight Vital Signs Heart Rate Blood Pressure Flow Rate Transient Pressure Aneurysm Properties Tissue Properties Wall Thickness Risk Factors Medical Images Medications Aneurysm Rupture Profile Morphology Profile Haemodynamic Profile Mechanobiological Profile Prediction Uncertainity PatientAvatar Updated RISK Patients Patient Avatar Disease Simulation Workflow Patient Avatar updatedSystemic Factors Gene Expression Profile RISK http://www.vph-share.eu/ VPH-Share @neurist Aneurysm Morphology Workflow
  15. 15. Implementation in VPH-Share The @neurIST morphological workflow specification in Taverna http://www.vph-share.eu/
  16. 16. Make Deploy Save and share Taverna Workbench VPH-Share Taverna Player Commandline Taverna (batch) Workflow repository Integrate and run Analysis portals Taverna Server
  17. 17. Taverna Workbench • Desktop application • GUI • Plug-in Framework • Themed editions • Intermediate results views • Search for Web Services in catalogues • Search and publish to myExperiment
  18. 18. Web apps to create and run workflows Taverna Online • Dr Vadim Surpin and Vitaly Sharanutsa – Institute for Information Transmission Problems of Russian Academy of Sciences (IITP RAS) • An online, in-browser application for assembling and running Taverna Workflows over a HPC platform
  19. 19. Web apps to create and run workflows Service Chaining Editor Pete Walker et al Plymouth Marine Laboratory For chaining OGC Web Processing Service geospatial Web Services
  20. 20. Desktop Client http://www.xworx.org/ Data Centric Interface BIFI (Beautiful Interfaces for Inputs) Taverna Workbench Plug-in, GUI definition language
  21. 21. Taverna Server family • Taverna Server – Multiple clients, Multi-user – Local and large scale infrastructures – Site Replication • Taverna Server Amazon Image – Can have local tools and services (e.g. R) – Multiple instances in Amazon Cloud and as required, for multiple users/uses and different security scenarios • Taverna Virtual Machine • Taverna Command Line • Bundled Servers, Services and Tools
  22. 22. Interacting with a workflow • Many tasks need user interaction • A workflow on a server does not need to be “press a button and wait” – VPH-Share opens a VNC connection to the spawned instance. • Taverna Interaction Service – Users interact with a workflow (wherever it is running) in a Web browser. – Interaction Service Plug-in in workbench
  23. 23. iPython integration https://www.youtube.com/watch?v=QVQwSOX5S08http://goo.gl/hm0qCN
  24. 24. Analysis Portals https://www.youtube.com/watch?v=s3D8JXc-tSM
  25. 25. – Find and share workflows (Taverna, RapidMiner, Kepler, Galaxy, Trident, Vistrails, etc) – Track updates of workflows – Social curation: Comments, tags, stars – Themed groups of users (projects, domain) – Organize packs of related data (source files, results) Over 7500 members, 300 groups, 2500 workflows, 600 files and 300 packs (research objects)
  26. 26. VPH-Share Models of Human Physiology Eagle Genomics Next Generation Sequencing based Patient Diagnostics Astronomy & HelioPhysics Document Preservation Digitisation Systems Biology OpenTox Project Chemistry Development Kit Drug Toxicity Ecological Niche Modelling Population Modelling Meta- genomics Phylo- genetics • Data cleaning • Data movement • Data retrieval and annotation • Data analysis • Data mining • knowledge management • Data curation and data warehouse population • Data visualisation • Parameter sweeps over simulations Drug discovery, small molecules, targets, compounds OpenPHACTS
  27. 27. Open source development • Taverna is open source software (LGPL) – https://github.com/taverna/ – License allows integration in closed-source products • Open development – Developer documentation and tutorials – Public mailing lists, issue trackers, wiki – Contributors from around the world • Taverna Plugins – APIs and plugin system • Applying to join the Apache Foundation
  28. 28. Summary • Taverna Suite for interactive and batch workflows • Flexible Plug-ins and Flexibly Plugged-in • Themed Taverna • Moving to the Apache Foundation • We welcome collaboration/contribution • http://www.taverna.org.uk
  29. 29. Integrating with Taverna • “anything” can be extended by plugins: – Service types – Service discovery – Menus and toolbars – File types (e.g. SVG, PDF, CSV) – Complete views/perspectives • Documentation and tutorials for plugin developers • Installable/updatable from plugin sites • Many plugins gets included in Taverna Core or domain-specific editions (e.g. AstroTaverna++ became Taverna Astronomy edition) • AstroTaverna plugin • OAuth plugin • Taverna PROV plugin • VAMDC plugin • BIFI plugin • VPH-Share plugin • Interaction plugin • XPath plugin • REST plugin • BioCatalogue plugin • PBS plugin • SADI plugin • External Tools plugin • UNICORE plugin • CDK plugin • caGrid plugin • XWS plugin • gLite plugin • WPS plugin • ... http://dev.mygrid.org.uk/wiki/display/developer/Creating+plugins+for +Taverna+2

×