Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

2017-07-22 Common Workflow Language Viewer

153 views

Published on

Authors: Mark Robinson, Stian Soiland-Reyes, Michael Crusoe, Carole Goble
Presented at BOSC 2017, ISMB/ECCB 2017, Prague.
Presentation as HTML: https://slides.com/markrobinson/cwlviewer
cwlviewer: https://view.commonwl.org

Published in: Software
  • Be the first to comment

  • Be the first to like this

2017-07-22 Common Workflow Language Viewer

  1. 1. Common Workflow Language Viewer https://view.commonwl.org Mark Robinson Stian Soiland-Reyes Michael Crusoe Carole Goble http://slides.com/markrobinson/cwlviewer http://orcid.org/0000-0002-8184-7507 http://orcid.org/0000-0001-9842-9718 http://orcid.org/0000-0002-2961-9670 http://orcid.org/0000-0003-1219-2137 @soilandreyes These slides are licensed under a Creative Commons Attribution 4.0 International License 2017-07-22 BOSC 2017 ISMB/ECCB 2017, Prague
  2. 2. Difficulties in sharing CWL Workflow spread over multiple text files https://github.com/common-workflow-language/ workflows/tree/master/workflows/lobSTR/lobSTR-workflow.cwl
  3. 3. Difficulties in sharing CWL Often missing: Detailed description (what, how) Provenance (who, where, when) Versioning (which) #!/usr/bin/env cwl-runner cwlVersion: v1.0 class: Workflow inputs: #SCWRL scw_script: File scw_input_pdb_path: File scw_output_pdb_path: string scw_mutation: string scw_scwrl_path: string scw_log_path: string scw_error_path: string #PDB2GMX p2g_script: File p2g_output_gro_path: string p2g_output_top_path: string p2g_output_itp_path: string p2g_output_top_tar_path: string p2g_water_type: string p2g_force_field: string p2g_ignh: string p2g_gmx_path: string p2g_log_path: string p2g_error_path: string # ... We need better incentives!
  4. 4. #!/usr/bin/env cwl-runner cwlVersion: "cwl:draft-3" class: Workflow inputs: - id: p1 type: - "null" - type: array items: File description: list of files containing the first end of paired end reads in fasta or fastq format - id: p2 type: - "null" - type: array items: File description: list of files containing the second end of paired end reads in fasta or fastq format - id: output_prefix type: string description: prefix for output files. will output prefix.aligned.bam and prefix.aligned.stats - id: reference type: File description: "lobSTR's bwa reference files" - id: rg-sample type: string description: Use this in the read group SM tag - id: rg-lib type: string description: Use this in the read group LB tag - id: strinfo type: File description: File containing statistics for each STR. - id: noise_model type: File description: File to read noise model parameters from (.stepmodel) secondaryFiles: - "^.stuttermodel" outputs: - id: bam type: File source: "#samindex/bam_with_bai" - id: bam_stats type: File source: "#lobSTR/bam_stats" - id: vcf type: File source: "#allelotype/vcf" - id: vcf_stats type: File source: "#allelotype/vcf_stats" hints: - class: DockerRequirement dockerLoad: https://workbench.qr1hi.arvadosapi.com/collections/download/qr1hi-4zz18-x2ae13tsx5jqg8d/1nduktd8dpvhdpgsva82lje0i710kgzb6rttks5jldx7s2y7k9/7e0c0ae3bf4e70442f9b8eee816ec23426d9e1169a2925316e5c932745e21613.tar dockerImageId: 7e0c0ae3bf4e70442f9b8eee816ec23426d9e1169a2925316e5c932745e21613 steps: - id: lobSTR run: lobSTR-tool.cwl inputs: - { id: p1, source: "#p1" } - { id: p2, source: "#p2" } - { id: output_prefix, source: "#output_prefix" } - { id: reference, source: "#reference" } - { id: rg-sample, source: "#rg-sample" } - { id: rg-lib, source: "#rg-lib" } outputs: - { id: bam } - { id: bam_stats } - id: samsort run: samtools-sort.cwl inputs: - { id: input, source: "#lobSTR/bam" } - { id: output_name, default: "aligned.sorted.bam" } outputs: - { id: output_file } - id: samindex run: samtools-index.cwl inputs: - { id: input, source: "#samsort/output_file" } outputs: - { id: bam_with_bai } - id: allelotype run: allelotype.cwl inputs: - { id: bam, source: "#samindex/bam_with_bai" } - { id: reference, source: "#reference" } - { id: output_prefix, source: "#output_prefix" } - { id: noise_model, source: "#noise_model" } - { id: strinfo, source: "#strinfo" } outputs: - { id: vcf } - { id: vcf_stats } Workflow Inputs Workflow Outputs reference allelotype lobSTR rg-samplep1p2 output_prefixrg-lib strinfonoise_model vcfvcf_statsbam_stats bam samsort samindex "aligned.sorted.bam" Visualisation = quick and easy understanding
  5. 5. CWL Viewer https://view.commonwl.org
  6. 6. View a Workflow https://view.commonwl.org/workflows/github.com/common-workflow-language/ workflows/tree/master/workflows/lobSTR/lobSTR-workflow.cwl
  7. 7. Interactivity https://view.commonwl.org/workflows/github.com/ProteinsWebTeam/ebi- metagenomics-cwl/tree/master/tools/rRNA_selection.cwl
  8. 8. Download and Share Visualisation Images (various formats) GraphViz DOT source Research Object Bundle/ Origin Information
  9. 9. COMMON WORKFLOW LANGUAGE A Research Object bundles and relates digital resources of a scientific experiment or investigation: Data used and results produced in experimental study Methods employed to produce and analyse that data Provenance and settings for the experiments People involved in the investigation Annotations about these resources, to improve understanding and interpretation
  10. 10. bundle.zip ├── mimetype ├── .ro │ └── manifest.json └── workflow ├── completeWorkflow.cwl ├── concatFasta.cwl ├── fastqc.cwl ├── generateRReports.cwl ├── qiimeAddMetadata.cwl ├── readPair.yml ├── uparseChimeraCheck.cwl ├── uparseDerepWorkAround.cwl ├── uparseFastqMerge.cwl ├── uparseFilter.cwl └── uparseTruncateLength.cwl { "@context" : [ "https://w3id.org/bundle/context" ], "id" : "/", "manifest" : [ "manifest.json" ], "createdOn" : "2017-02-27T11:48:54.986Z", "createdBy" : { "uri" : "http://view.commonwl.org", "name" : "Common Workflow Language Viewer" }, "authoredBy" : [ { "uri" : "https://github.com/grbot", "name" : "Gerrit Botha" }, { "uri" : "https://github.com/mr-c", "name" : "Michael R. Crusoe" }, { "name" : "Ubuntu" }, { "uri" : "https://github.com/mepstein-gh", "name" : "Milt Epstein" }, { "uri" : "https://github.com/ShakunBaichoo", "name" : "ShakunBaichoo" } ], "importedFrom" : "https://github.com/h3abionet/h3abionet16S/tree/70498847fd6bfc63f44c59c748233cac7f8dd85d/workfl "importedOn" : "2017-02-27T11:48:54.986Z", "importedBy" : { "uri" : "http://view.commonwl.org", "name" : "Common Workflow Language Viewer" }, "aggregates" : [ { "uri" : "/workflow/completeWorkflow.cwl", "mediatype" : "text/x-yaml", "createdOn" : "2017-02-27T11:48:55.21Z", "authoredBy" : [ { "uri" : "https://github.com/grbot", "name" : "Gerrit Botha" }, { "uri" : "https://github.com/mr-c", "name" : "Michael R. Crusoe" }, { "uri" : "https://github.com/mepstein-gh", "name" : "Milt Epstein" }, { "uri" : "https://github.com/ShakunBaichoo", "name" : "ShakunBaichoo" } ], "retrievedFrom" : "https://github.com/h3abionet/h3abionet16S/blob/70498847fd6bfc63f44c59c748233cac7f8dd85d/wor "conformsTo" : "https://w3id.org/cwl/v1.0", "bundledAs" : { "uri" : "urn:uuid:fbc4f48a-b111-4e1b-b6b9-d6a97c7c7f90", "folder" : "/workflow/" } }, { "uri" : "/workflow/concatFasta.cwl", "mediatype" : "text/x-yaml", Provenance, attribution Who made which part of the workflow when
  11. 11. CWL Viewer Research Objects
  12. 12. BOSC 2017 Birds of a Feather Today 12:20-13:40 BoF #2: Common Workflow Language community meetup Farah Zaib Khan & Stian Soiland-Reyes: ResearchObject.org & CWL integration COMMON WORKFLOW LANGUAGE
  13. 13. Discovery of Workflows https://view.commonwl.org/workflows
  14. 14. In the Works! JSON API Use of the cwltool reference implementation for parsing Input labelling More import options And More! Workflow Inputs Workflow Outputs p2 lobSTR p2 rg-lib rg-lib p1 p1 reference reference allelotype reference output_prefix output_prefix output_prefix noise_model noise_model rg-sample rg-sample strinfo strinfo vcf_stats vcfbambam_stats samsort input samindex bam input "aligned.sorted.bam" output_name cwl-svg
  15. 15. https://view.commonwl.org/ Join us! https://github.com/ common-workflow-language/cwlviewer Poster (A-124): https://doi.org/10.7490/f1000research.1114375.1

×