Authors: Mark Robinson, Stian Soiland-Reyes, Michael Crusoe, Carole Goble Presented at BOSC 2017, ISMB/ECCB 2017, Prague. Presentation as HTML: https://slides.com/markrobinson/cwlviewer cwlviewer: https://view.commonwl.org

1. Common Workflow
Language Viewer
https://view.commonwl.org
Mark Robinson
Stian Soiland-Reyes
Michael Crusoe
Carole Goble
http://slides.com/markrobinson/cwlviewer
http://orcid.org/0000-0002-8184-7507
http://orcid.org/0000-0001-9842-9718
http://orcid.org/0000-0002-2961-9670
http://orcid.org/0000-0003-1219-2137
@soilandreyes
These slides are licensed under a
Creative Commons Attribution 4.0 International License
2017-07-22 BOSC 2017
ISMB/ECCB 2017, Prague

2. Difficulties in sharing CWL
Workflow spread over multiple text files
https://github.com/common-workflow-language/
workflows/tree/master/workflows/lobSTR/lobSTR-workflow.cwl

3. Difficulties in sharing CWL
Often missing:
Detailed description (what, how)
Provenance (who, where, when)
Versioning (which)
#!/usr/bin/env cwl-runner
cwlVersion: v1.0
class: Workflow
inputs:
#SCWRL
scw_script: File
scw_input_pdb_path: File
scw_output_pdb_path: string
scw_mutation: string
scw_scwrl_path: string
scw_log_path: string
scw_error_path: string
#PDB2GMX
p2g_script: File
p2g_output_gro_path: string
p2g_output_top_path: string
p2g_output_itp_path: string
p2g_output_top_tar_path: string
p2g_water_type: string
p2g_force_field: string
p2g_ignh: string
p2g_gmx_path: string
p2g_log_path: string
p2g_error_path: string
# ...
We need better incentives!

4. #!/usr/bin/env cwl-runner
cwlVersion: "cwl:draft-3"
class: Workflow
inputs:
- id: p1
type:
- "null"
- type: array
items: File
description: list of files containing the first end of paired end reads in fasta or fastq format
- id: p2
type:
- "null"
- type: array
items: File
description: list of files containing the second end of paired end reads in fasta or fastq format
- id: output_prefix
type: string
description: prefix for output files. will output prefix.aligned.bam and prefix.aligned.stats
- id: reference
type: File
description: "lobSTR's bwa reference files"
- id: rg-sample
type: string
description: Use this in the read group SM tag
- id: rg-lib
type: string
description: Use this in the read group LB tag
- id: strinfo
type: File
description: File containing statistics for each STR.
- id: noise_model
type: File
description: File to read noise model parameters from (.stepmodel)
secondaryFiles:
- "^.stuttermodel"
outputs:
- id: bam
type: File
source: "#samindex/bam_with_bai"
- id: bam_stats
type: File
source: "#lobSTR/bam_stats"
- id: vcf
type: File
source: "#allelotype/vcf"
- id: vcf_stats
type: File
source: "#allelotype/vcf_stats"
hints:
- class: DockerRequirement
dockerLoad: https://workbench.qr1hi.arvadosapi.com/collections/download/qr1hi-4zz18-x2ae13tsx5jqg8d/1nduktd8dpvhdpgsva82lje0i710kgzb6rttks5jldx7s2y7k9/7e0c0ae3bf4e70442f9b8eee816ec23426d9e1169a2925316e5c932745e21613.tar
dockerImageId: 7e0c0ae3bf4e70442f9b8eee816ec23426d9e1169a2925316e5c932745e21613
steps:
- id: lobSTR
run: lobSTR-tool.cwl
inputs:
- { id: p1, source: "#p1" }
- { id: p2, source: "#p2" }
- { id: output_prefix, source: "#output_prefix" }
- { id: reference, source: "#reference" }
- { id: rg-sample, source: "#rg-sample" }
- { id: rg-lib, source: "#rg-lib" }
outputs:
- { id: bam }
- { id: bam_stats }
- id: samsort
run: samtools-sort.cwl
inputs:
- { id: input, source: "#lobSTR/bam" }
- { id: output_name, default: "aligned.sorted.bam" }
outputs:
- { id: output_file }
- id: samindex
run: samtools-index.cwl
inputs:
- { id: input, source: "#samsort/output_file" }
outputs:
- { id: bam_with_bai }
- id: allelotype
run: allelotype.cwl
inputs:
- { id: bam, source: "#samindex/bam_with_bai" }
- { id: reference, source: "#reference" }
- { id: output_prefix, source: "#output_prefix" }
- { id: noise_model, source: "#noise_model" }
- { id: strinfo, source: "#strinfo" }
outputs:
- { id: vcf }
- { id: vcf_stats }
Workflow Inputs
Workflow Outputs
reference
allelotype
lobSTR
rg-samplep1p2 output_prefixrg-lib strinfonoise_model
vcfvcf_statsbam_stats bam
samsort
samindex
"aligned.sorted.bam"
Visualisation = quick and easy understanding

5. CWL Viewer
https://view.commonwl.org

6. View a Workflow
https://view.commonwl.org/workflows/github.com/common-workflow-language/
workflows/tree/master/workflows/lobSTR/lobSTR-workflow.cwl

7. Interactivity
https://view.commonwl.org/workflows/github.com/ProteinsWebTeam/ebi-
metagenomics-cwl/tree/master/tools/rRNA_selection.cwl

8. Download and Share
Visualisation Images
(various formats)
GraphViz DOT source
Research Object Bundle/
Origin Information

9. COMMON
WORKFLOW
LANGUAGE
A Research Object bundles and relates digital
resources of a scientific experiment or
investigation:
Data used and results produced in experimental
study
Methods employed to produce and analyse that
data
Provenance and settings for the experiments
People involved in the investigation
Annotations about these resources, to improve
understanding and interpretation

10. bundle.zip
├── mimetype
├── .ro
│ └── manifest.json
└── workflow
├── completeWorkflow.cwl
├── concatFasta.cwl
├── fastqc.cwl
├── generateRReports.cwl
├── qiimeAddMetadata.cwl
├── readPair.yml
├── uparseChimeraCheck.cwl
├── uparseDerepWorkAround.cwl
├── uparseFastqMerge.cwl
├── uparseFilter.cwl
└── uparseTruncateLength.cwl
{
"@context" : [ "https://w3id.org/bundle/context" ],
"id" : "/",
"manifest" : [ "manifest.json" ],
"createdOn" : "2017-02-27T11:48:54.986Z",
"createdBy" : {
"uri" : "http://view.commonwl.org",
"name" : "Common Workflow Language Viewer"
},
"authoredBy" : [ {
"uri" : "https://github.com/grbot",
"name" : "Gerrit Botha"
}, {
"uri" : "https://github.com/mr-c",
"name" : "Michael R. Crusoe"
}, {
"name" : "Ubuntu"
}, {
"uri" : "https://github.com/mepstein-gh",
"name" : "Milt Epstein"
}, {
"uri" : "https://github.com/ShakunBaichoo",
"name" : "ShakunBaichoo"
} ],
"importedFrom" : "https://github.com/h3abionet/h3abionet16S/tree/70498847fd6bfc63f44c59c748233cac7f8dd85d/workfl
"importedOn" : "2017-02-27T11:48:54.986Z",
"importedBy" : {
"uri" : "http://view.commonwl.org",
"name" : "Common Workflow Language Viewer"
},
"aggregates" : [ {
"uri" : "/workflow/completeWorkflow.cwl",
"mediatype" : "text/x-yaml",
"createdOn" : "2017-02-27T11:48:55.21Z",
"authoredBy" : [ {
"uri" : "https://github.com/grbot",
"name" : "Gerrit Botha"
}, {
"uri" : "https://github.com/mr-c",
"name" : "Michael R. Crusoe"
}, {
"uri" : "https://github.com/mepstein-gh",
"name" : "Milt Epstein"
}, {
"uri" : "https://github.com/ShakunBaichoo",
"name" : "ShakunBaichoo"
} ],
"retrievedFrom" : "https://github.com/h3abionet/h3abionet16S/blob/70498847fd6bfc63f44c59c748233cac7f8dd85d/wor
"conformsTo" : "https://w3id.org/cwl/v1.0",
"bundledAs" : {
"uri" : "urn:uuid:fbc4f48a-b111-4e1b-b6b9-d6a97c7c7f90",
"folder" : "/workflow/"
}
}, {
"uri" : "/workflow/concatFasta.cwl",
"mediatype" : "text/x-yaml",
Provenance, attribution
Who made which part of the workflow
when

11. CWL Viewer Research Objects

12. BOSC 2017
Birds of a
Feather
Today 12:20-13:40
BoF #2: Common Workflow Language
community meetup
Farah Zaib Khan & Stian Soiland-Reyes:
ResearchObject.org & CWL integration
COMMON
WORKFLOW
LANGUAGE

13. Discovery of Workflows
https://view.commonwl.org/workflows

14. In the Works!
JSON API Use of the cwltool reference
implementation for parsing
Input labelling
More import options
And More!
Workflow Inputs
Workflow Outputs
p2
lobSTR
p2
rg-lib
rg-lib
p1
p1
reference
reference
allelotype
reference
output_prefix
output_prefix
output_prefix
noise_model
noise_model
rg-sample
rg-sample
strinfo
strinfo
vcf_stats vcfbambam_stats
samsort
input
samindex
bam
input
"aligned.sorted.bam"
output_name
cwl-svg

15. https://view.commonwl.org/
Join us!
https://github.com/
common-workflow-language/cwlviewer
Poster (A-124):
https://doi.org/10.7490/f1000research.1114375.1