IMPACT/myGrid Hackathon - Taverna Roadmap

2,920 views

Published on

IMPACT/myGrid Hackathon - Taverna Roadmap with Shoaib Sufi - Manchester 14th November 2011

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,920
On SlideShare
0
From Embeds
0
Number of Embeds
1,946
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Some are specific but have generic use possible Others generic use with a specific customer/project
  • IMPACT/myGrid Hackathon - Taverna Roadmap

    1. 1. Taverna Roadmap Shoaib Sufi myGrid Project Manager
    2. 2. Sources of Influence <ul><li>Projects </li></ul><ul><ul><li>SCAPE – Digital preservation </li></ul></ul><ul><ul><li>BioVel – Biodiversity science </li></ul></ul><ul><ul><li>CA4LS – Next Generation Sequencing </li></ul></ul><ul><ul><li>Wf4ever – Workflow preservation Astronomy & Bioinformatics focus </li></ul></ul><ul><ul><li>e-Lico – Data mining – (Kidney function data) </li></ul></ul><ul><ul><li>NeISS – Social Simulation </li></ul></ul><ul><li>Collaboration </li></ul><ul><ul><li>IMPACT – Digital preservation </li></ul></ul><ul><ul><li>HELIO – Solar physics </li></ul></ul><ul><ul><li>Taverna - Galaxy – Bioinformatics </li></ul></ul><ul><ul><li>ViBRANT – Biodiversity </li></ul></ul><ul><li>User communities </li></ul><ul><ul><li>Super users requests (experts e.g. Andrea Wiggins) </li></ul></ul><ul><ul><li>User requests </li></ul></ul><ul><li>Carole </li></ul><ul><ul><li>Feedback from conference attendance </li></ul></ul><ul><li>Developer community </li></ul><ul><ul><li>What are other projects doing </li></ul></ul><ul><ul><li>Developer requests and contributions (integration) </li></ul></ul><ul><li>Useful to do / take into account </li></ul><ul><ul><li>Clouds </li></ul></ul><ul><ul><li>Semantic Web </li></ul></ul><ul><ul><li>Other promising or proven technologies </li></ul></ul>
    3. 3. Taverna 2.3 Server Release 1 (November 2011) <ul><li>Hardened, debugged and tested </li></ul><ul><li>Improved large input and output data handling </li></ul><ul><li>Security support </li></ul><ul><ul><li>User authentication to the Server </li></ul></ul><ul><ul><li>Secure service invocation from inside workflows </li></ul></ul><ul><ul><li>Workflow run access control </li></ul></ul><ul><li>Improved job management and accounting trails </li></ul><ul><li>Notification of workflow finishing using various mechanisms </li></ul>
    4. 4. Taverna 3 <ul><li>Why – Standards compliance (OSGi), aid reuse </li></ul><ul><li>Taverna 3 Platform OSGi beta and Command Line Tool (January 2012) </li></ul><ul><ul><li>Re-engineering the Taverna Engine using OSGi </li></ul></ul><ul><ul><li>Support programmatic access to the Taverna Core </li></ul></ul><ul><ul><li>New Run/Execution Manager and Edits API </li></ul></ul><ul><ul><li>Embeddable in applications for executing workflows </li></ul></ul><ul><li>Taverna 3 Platform OSGi and Command Line Tool (July 2012) </li></ul><ul><ul><li>Stabilized and hardened </li></ul></ul><ul><ul><li>Plugin-enabled </li></ul></ul><ul><li>Taverna 3 Platform OSGi and Workbench alpha (July 2012) </li></ul><ul><ul><li>Based on the Taverna 3 Platform OSGi </li></ul></ul><ul><li>Taverna 3 Platform OSGi and Server beta (July 2012) </li></ul><ul><ul><li>Based on the Taverna 3 Platform OSGi </li></ul></ul>
    5. 5. Draft Taverna 3 features <ul><li>Better user-interaction during workflow run </li></ul><ul><ul><li>- Improved interaction with visualizing tools and services </li></ul></ul><ul><ul><ul><li>‘ Blackboard’ support </li></ul></ul></ul><ul><li>Improved workflow diagram </li></ul><ul><ul><li>Mark as boring </li></ul></ul><ul><ul><li>More obvious looping & iteration strategy </li></ul></ul><ul><li>Data </li></ul><ul><ul><li>JSON related services (create, search, extract) (in 2.x also) </li></ul></ul><ul><ul><li>XML related service (create via templates) (in 2.x also) </li></ul></ul><ul><ul><li>Table support (structured objects) </li></ul></ul><ul><li>Services </li></ul><ul><ul><li>Choice of languages for in-workflow-programming via Java Scripting framework </li></ul></ul><ul><ul><ul><li>A better ‘bean shell’ (e.g. python or other languages supported by JVM) </li></ul></ul></ul><ul><li>Components (semantically annotated workflows as pipeline steps) </li></ul><ul><ul><li>- Components creation/deletion </li></ul></ul><ul><ul><li>- Components registry </li></ul></ul><ul><li>Enhanced execution </li></ul><ul><ul><li>Ability to change WSDL bindings </li></ul></ul><ul><ul><li>Intelligent re-run/resume of workflows (use results before error occurred) </li></ul></ul><ul><ul><li>Tarpit mitigation – i.e. XSD caching (2.x also) </li></ul></ul>
    6. 6. Taverna 3 features continued … <ul><li>Improved Provenance </li></ul><ul><ul><li>More efficient provenance capture </li></ul></ul><ul><ul><li>More detailed provenance capture </li></ul></ul><ul><ul><li>Export of Provenance in OPM and W3 PROV serialisation </li></ul></ul><ul><li>Research Object support </li></ul><ul><ul><li>via SCUFL 2 Data bundle: </li></ul></ul><ul><ul><ul><li>Workflow </li></ul></ul></ul><ul><ul><ul><li>Provenance </li></ul></ul></ul><ul><ul><ul><li>Data </li></ul></ul></ul><ul><ul><ul><ul><li>In </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Out </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Intermediate </li></ul></ul></ul></ul><ul><ul><ul><li>Manifest </li></ul></ul></ul><ul><li>Control flow </li></ul><ul><ul><li>Branching support (if/else) </li></ul></ul><ul><ul><li>Better looping (explicit while) </li></ul></ul>
    7. 7. Taverna 3 features continued(2) … <ul><li>Domain specific support (from projects req’s) </li></ul><ul><ul><li>Astronomy </li></ul></ul><ul><ul><ul><li>VOTable visualisation support (Workbench) </li></ul></ul></ul><ul><ul><ul><li>VOTable local workers support (Engine) </li></ul></ul></ul><ul><ul><ul><li>Opens way for Table support +XML template use case </li></ul></ul></ul><ul><ul><ul><li>SAMP support (Workbench) local blackboard </li></ul></ul></ul><ul><ul><li>Biodiversity </li></ul></ul><ul><ul><ul><li>Asynchronous input support </li></ul></ul></ul><ul><ul><ul><ul><li>Sensor data </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Relevant to other instruments producing continuous data </li></ul></ul></ul></ul></ul><ul><li>Security </li></ul><ul><ul><li>Single Sign On </li></ul></ul><ul><li>Server </li></ul><ul><ul><li>Pause/resume </li></ul></ul><ul><ul><li>Fine grained monitoring and feedback for a running workflow. </li></ul></ul><ul><ul><ul><li>Where is it upto </li></ul></ul></ul>
    8. 8. SCUFL2 – the new workflow format <ul><li>SCUFL2 Language Specification (January 2012) </li></ul><ul><ul><li>Taverna workflow language specification more like SCUFL from Taverna 1.7.x to replace the current t2flow serialisation format </li></ul></ul><ul><ul><li>Easier for third party use </li></ul></ul><ul><ul><li>A packaged format with annotation support </li></ul></ul><ul><li>SCUFL2 Tools beta (January 2012) </li></ul><ul><ul><li>Tools for conversion from t2flow to SCUFL2 </li></ul></ul><ul><li>SCUFL2 Tools Release (July 2012) </li></ul><ul><ul><li>A stabilised and fully tested version </li></ul></ul><ul><ul><li>Plugin for the Taverna Workbench 2.3 </li></ul></ul><ul><ul><li>Plugin for the Taverna Server 2.3 </li></ul></ul><ul><li>Other SCULF2 tooling </li></ul><ul><ul><li>workflows to Hadoop/MapReduce (SCAPE project) </li></ul></ul><ul><ul><li>Workflows to Wf4ever abstract workflow format </li></ul></ul><ul><ul><ul><li>Processers and links (no activities) – Skeleton workflow </li></ul></ul></ul><ul><ul><ul><li>Conversion from abstract format to SCULF2 (won’t execute but act as template) </li></ul></ul></ul>
    9. 9. Cloudy with a chance of Taverna <ul><li>Taverna Infrastructure Virtual Appliance </li></ul><ul><li>Taverna Server Virtual Machine (VM) (December 2011) </li></ul><ul><ul><li>Including Taverna 2.3 Server and management interface </li></ul></ul><ul><li>Amazon Machine Interface (AMI) (December 2011) </li></ul><ul><ul><li>Including Taverna 2.3 Server and management interface </li></ul></ul>
    10. 10. KISS – Taverna keeping it simple <ul><li>(December 2011 alpha) </li></ul><ul><li>Web based interface </li></ul><ul><li>Connecting workflows </li></ul><ul><li>Hides complex iterations/looping/config in ‘components’ </li></ul><ul><li>Build ‘component’ workflows in Taverna Workbench and then deploy in KISS </li></ul><ul><li>Simple install and one config file containing workflow definitions for a particular domain/instance </li></ul><ul><li>Uses Wireit </li></ul><ul><li>Will shortly have a Baclava data viewer </li></ul><ul><li>Alpha (uses command line) </li></ul><ul><li>Multi user support Q1 2012 </li></ul><ul><ul><li>Will use Taverna Server </li></ul></ul>
    11. 12. Plugins <ul><li>Data Mining plugin (December 2011) </li></ul><ul><ul><li>Makes all the RapidMiner (www.rapidminer.com) services available in Taverna </li></ul></ul><ul><ul><ul><li>more than 500 operators for data integration and transformation, data mining, evaluation, and visualization </li></ul></ul></ul><ul><ul><li>Automatic workflow building via metadata and goals </li></ul></ul><ul><li>SPARQL plugin (2.x and 3) </li></ul><ul><ul><li>Query RDF data sources (e.g Linked Data) </li></ul></ul>

    ×